Skip to content

Posts from the ‘Unix’ Category


PowerDNS: Configuring and Running Authoritative & Recursor Servers In The Same Host.

This article gives a quick introduction of setting up PowerDNS (pdns) Authoritative and Recursor servers, and also demonstrates a scenario that how we need to use both in the same host.


DNS has two type of server operations: Authoritative and Recursive. Most common use of DNS servers is authoritative. For those who don’t know the difference, here is a good introduction. Both are for different purposes, also both types of server can be operated individually and cooperatively under the same host. Read more »


Extract a list of symbols from stock exchange indices

As far as I know there is no free web service that can produce a single page of stock symbols from a stock exchange. Moreover, for webpages that produce stock symbols, all of them only output with pagination. However, the easiest webpage to extract the symbols list is the Yahoo finance. This article shows how to extract a full list of symbols with a few repeated commands.

For Yahoo Finance, each page list approximately 50 symbols and it takes 3 calls to get all the symbols of FTSE 100 stock exchange. Here are the commands:

# Return the first 49 symbols
curl -s "" | grep -o '>[A-Z]*\.L</a>' | sed -n "s/>\(.*\)<\/a>/\1/p"
# Return the next 50 symbols
curl -s "" | grep -o '>[A-Z]*\.L</a>' | sed -n "s/>\(.*\)<\/a>/\1/p"
# Return the final symbol (last page)
curl -s "" | grep -o '>[A-Z]*\.L</a>' | sed -n "s/>\(.*\)<\/a>/\1/p"

For Dow Jones Industrial Average, it returns 30 symbols

curl -s "" | grep -o 'q?s=[A-Z]*">[A-Z]*' | sed -n 's/.*=.*">\([A-Z]*\)<\/a>/\1/p'

For Nasdaq Composite, it is the same as above except changing the cp query to s=%5IXIC and repeat for many pages with parameter c. Alternatively, you can use this link to download the whole list.


Emacs command line to indent Javascript code and convert tabs to spaces

Part of my project checkin script is to automatically apply coding style and converting tabs to spaces. Here is the command line:

emacs -batch file.js --eval "(custom-set-variables '(js-indent-level 2))"  -f mark-whole-buffer -f indent-region -f mark-whole-buffer -f untabify -f save-buffer -kill

Change the value of js-indent-level to any religion of white spaces.


What to do when you mess up the setuid bit of all the files from root directory?

unixYou can mess up the access mode of a file if you do a ‘chown -R’ or ‘chmod -R’ over the wrong directory. Well, I did!! This will screw up the setuid bit which can lead to permission problem on some essential utilities, such as mount, dhclient, etc. Read more »


Making the most of unix commands when programming (Part 2) – the awk command

unixAwk command is one of the most useful unix tools and yet the most (in my view) underrated. It can save you a lot of efforts writing any code.  Everyone uses a lot of grep in conjunction with programming but awk can do more than grep in some cases. Read more »


Handy unix commands for debugging

unixI spend most of my time in my job reading other peoples’ code or kernel source code than writing my own. These code can be scattered into many directories in multiple levels. I constantly have to do ‘find’ and ‘grep’ to trace variable and function names. Typing these commands manually can add up a lot of time. Read more »


Making the most of unix commands when programming (Part 1) – the ps command

unixWith the use of unix commands not only can reduce programming effort, but also if we utilise the them even more, less code is needed. Less code means less bugs. The well known unix tools philosophy is to ‘do one thing and do it well’. All we need is to make the tool doing well for our programs. In this article, I will give some examples in C and Php. Read more »


Guidelines on integrating complex unix command lines into programs

unixNeedless to say, the power of combining unix commands together can break any complicated tasks into challenging tricks. These commands are a great help in programming to offer great short cuts. Less code means less bugs. However, I have seen some code embedded with complex unix command lines used slightly to the extreme. Read more »