'jq' - Notes

References:
https://stedolan.github.io/jq/manual/
https://jqlang.github.io/jq/manual/

 

Keys With Hyphens

jq doesn't support array keys with a hyphen "-" in the name:

$ cat foo.json | jq '.result.devices.router1-rack1-pop1.vlans'
jq: error: rack1/0 is not defined at , line 1:
.result.devices.router1-rack1-pop1.vlans                        
jq: error: pop1/0 is not defined at , line 1:
.result.devices.router1-rack1-pop1.vlans                             
jq: 2 compile errors

They need to be escaped, not with slashes or quotes but, by using square brackets and double quotes inside single quotes:

$ cat foo.json | jq '.result.devices["router1-rack1-pop1"].vlans'
[
  {
    "name": "210",
    "vid": 210,
  }
]

 

Another option is to put all keys (including the one with the hyphen) in double quotes and the entire filter in single quotes:

$ jq '."objects"."object"[]."primary-key"."attribute"[]."value"' /tmp/inetnums
"128.0.0.0 - 128.0.7.255"

 

Filtering an Array of Objects

Use select to filter an array only for objects which have a specific value stored under a specific key:

jq '.[]|select(.data_type == "update")'

jq '.[]|select(.[].name=="inetnum" or .[].name=="status")'

 

Multiple Keys

Output multiple keys from the input data (a list of arrays in this case):

cat foo.json | jq '.[] | "\(.id), \(.name)"'
"1, foo"
"2, bar"
"3, baz"

 

Slurping

In the following example, the first jq command takes a list of objects fetched by cURL and filters based on the value of key "data_type". The output is each object with this k/v pair, but jq outputs them as individual objects, they are no longer in a single list of all objects. This means that the second jq command has to use -s which will read in all JSON text first, them combine it into a single large object/list, then apply the provided filters:

curl -s https://api.broker.bgpkit.com/v2/latest | jq '.[]|select(.data_type == "update")' | jq -s '.|sort_by(.rough_size)
...
  {
    "timestamp": "2022-08-15T07:45:00",
    "delay": 1225.893042,
    "collector_id": "route-views.amsix",
    "data_type": "update",
    "item_url": "http://archive.routeviews.org/route-views.amsix/bgpdata/2022.08/UPDATES/updates.20220815.0745.bz2",
    "rough_size": 6710886,
    "exact_size": 0,
    "collector_url": "http://archive.routeviews.org/route-views.amsix/bgpdata"
  },
  {
    "timestamp": "2022-08-15T07:45:00",
    "delay": 1225.893042,
    "collector_id": "route-views5",
    "data_type": "update",
    "item_url": "http://archive.routeviews.org/route-views5/bgpdata/2022.08/UPDATES/updates.20220815.0745.bz2",
    "rough_size": 8388608,
    "exact_size": 0,
    "collector_url": "http://archive.routeviews.org/route-views5/bgpdata"
  },
  {
    "timestamp": "2022-08-15T07:45:00",
    "delay": 1225.893042,
    "collector_id": "route-views3",
    "data_type": "update",
    "item_url": "http://archive.routeviews.org/route-views3/bgpdata/2022.08/UPDATES/updates.20220815.0745.bz2",
    "rough_size": 8912896,
    "exact_size": 0,
    "collector_url": "http://archive.routeviews.org/route-views3/bgpdata"
  }
]

The output for the first command shows the problem:

...
{
  "timestamp": "2022-08-15T07:35:00",
  ...
}
{
  "timestamp": "2022-08-15T07:35:00",
  ...
}
{
  "timestamp": "2022-08-15T07:30:00",
  ...
}

 

Raw Output

Add -r to NOT add double quotes to strings:

$ jq '."items"[]."metadata"."labels"."kubernetes.io/hostname"' /tmp/nodes 
"nodepool-1"
"nodepool-2"

$ jq -r '."items"[]."metadata"."labels"."kubernetes.io/hostname"' /tmp/nodes 
nodepool-1
nodepool-2

 

Sort By

Top 20 most connected ASNs (by ASN count)

curl -s https://broker-latest.bgpkit.workers.dev/peers-all | jq '.|sort_by(.num_connected_asns)' | grep \"asn\" | uniq | tail -n 20
    "asn": 13004,
    "asn": 35280,
    "asn": 51185,
    "asn": 57463,
    "asn": 35280,
    "asn": 14840,
    "asn": 61573,
    "asn": 36236,
    "asn": 264479,
    "asn": 61573,
    "asn": 24482,
    "asn": 1828,
    "asn": 24115,
    "asn": 6939,
    "asn": 26162,
    "asn": 36236,
    "asn": 174,
    "asn": 13004,
    "asn": 26162,
    "asn": 6939,

 

---