jq JSON processor

2023-06-08 jq JSON yq YAML xq XML curl

We have a web tool for earned value management of internal projects. It is written in ASP.NET MVC and it also provides nice JSON API to retrieve data.

There is tasks url that returns all available tasks like this:

[
  {
    "Id": 1,
    ...
    "Assignee": "Joe Smith",
    "Name": "Helper 1.0.0",
    "Predicted Start": "6/1/2021",
    "Predicted End": "8/31/2022",
    "Completed": "8/22/2022",
    ...
  },
  {
    "Id": 2,
    ...
    "Assignee": "Roman Hubacek",
    "Name": "VM - Migration",
    "Predicted Start": "9/30/2022",
    "Predicted End": "10/31/2022",
    "Completed": "10/17/2022",
    ...
  }
  ...
]

I can obtain such information with command-line tools like curl and then apply filtering on the data using jq. For example

curl https://projects/api/tasks | jq ".[] | select(.Assignee == \"Roman Hubacek\" and .\"Team Tracker\" == \"2023 Tools milestones\")

The filter above picks all entries in the top-level array and filter out only my tasks for current year. The filters can be nicely chained, by adding

| [ .Name, .\"Predicted End\", .Completed ]

I got only fields I am interested in as an array of arrays. This form can be also nicely formatted into csv with handy @csv filter. In that case it is useful to put in the -r option that outputs raw strings and does not escape each entry. We might also want to add a verbatim header

Altogether the command line is

curl https://projects/api/tasks | jq -r "([\"Name\",\"Predicted End\", \"Completed\"] | @csv), (.[] | select(.Assignee == \"Roman Hubacek\" and .\"Team Tracker\" == \"2023 Tools milestones\") | [ .Name, .\"Predicted End\", .Completed ] | @csv)"

Result is following table

Name Predicted End Completed
Milestone 1 6/16/2023 6/22/2023
Milestone 2 7/31/2023  
Milestone 3 4/17/2023 4/19/2023
Milestone 4 3/3/2023  
Milestone 5 6/30/2023 3/13/2023
Milestone 6 2/3/2023 2/3/2023
Milestone 7 1/20/2023 1/19/2023
Milestone 8 6/30/2023  

It did not occurred before to me that JSON can be processed this easily from command line, but it works quite well.