"q" is a nifty command line tool if you would like to do some text file processing and lookups.

So what is it?

Basically SQL for csv/tsv (and other tabelated) files. It supports most of the basic queries and also allows to perform them over multiple files (supports joining).

How does it work? You can imagine a file with tabelated data as a single table of a database. Just type in the command prompt:

Basic Query Example
$ q "select * from ./myfile.csv"

Columns are by default named c1, c2,...cn. The default delimiter is space (' ') but it can be changed with -d parameter. If your file contains a column name row, you can run q with a -H parameter. This will cause q to use the first row as column names. For example imagine a file names.csv:

names.csv
name, surname
Mike, Litoris
Ben, Dover
Seymour, Asses

Here are some examples of how the header parameter affects column names.

Query With/Without Column Headers
$ q -d , "select * from names.csv"
name, surname
Mike, Litoris
Ben, Dover
Seymour, Asses

$ q -d , -H "select * from names.csv"
Mike, Litoris
Ben, Dover
Seymour, Asses

$ q -d , "select name from names.csv"
query error: no such column: name

$ q -d , -H "select name from names.csv"
Mike
Ben
Seymour

$ q -d , -H "select c1 from names.csv"
query error: no such column: c1
Warning - There seems to be a "no such column" error, and -H (header line) exists. Please make sure that you are using the column names from the header line and not the default (cXX) column names

q also supports aliases and joining over multiple tables and files. Let’s add a file ages.csv:

ages.csv
surname, age
Litoris, 18
Dover, 20
Skung, 55
Dung, 31

Now, let’s see the inner join query

An Example Of Inner Join
$ q -d , -H "select n.name, n.surname, a.age from names.csv as n inner join ages.csv as a on n.surname=a.surname"

Mike,Litoris,18
Ben,Dover,20

q is also useable in a pipe. In such case, use - as table name. Behold:

Using q in a pipe
$ cat names.csv | q -d , -H "select name from -"
Mike
Ben
Seymour

And there is more: count, group by, order by, ...

q came in handy many times when I needed to examine logs on a remote machine and grep was not enough.