Use q For High IQ
"q" is a nifty command line tool if you would like to do some text file processing and lookups.
So what is it?
Basically SQL for csv/tsv (and other tabelated) files. It supports most of the basic queries and also allows to perform them over multiple files (supports joining).
How does it work? You can imagine a file with tabelated data as a single table of a database. Just type in the command prompt:
Columns are by default named c1, c2,...cn
.
The default delimiter is space (' ') but it can be changed with -d
parameter.
If your file contains a column name row, you can run q
with a -H
parameter.
This will cause q
to use the first row as column names.
For example imagine a file names.csv:
Here are some examples of how the header parameter affects column names.
q
also supports aliases and joining over multiple tables and files.
Let’s add a file ages.csv:
Now, let’s see the inner join query
q
is also useable in a pipe.
In such case, use -
as table name.
Behold:
And there is more: count, group by, order by, ...
q
came in handy many times when I needed to examine logs on a remote machine and grep
was not enough.