jq - JSON Processor

Key jq Features

jq '.field' data.json

jq '.[] | select(.age > 25)'

jq '{name, age}' data.json

jq '.' data.json

🔹 Basic jq Syntax

jq uses a simple dot notation to access JSON fields. The dot (.) represents the entire input, and you chain field names to navigate nested structures. This intuitive syntax makes JSON parsing straightforward and readable.

#!/bin/bash

# Sample JSON
json='{"name":"Alice","age":28,"city":"NYC"}'

# Extract single field
echo $json | jq '.name'

# Extract multiple fields
echo $json | jq '.name, .age'

# Pretty print entire JSON
echo $json | jq '.'

# Raw output (no quotes)
echo $json | jq -r '.name'

"Alice"
"Alice"
28
{
  "name": "Alice",
  "age": 28,
  "city": "NYC"
}
Alice

🔹 Working with Arrays

jq provides powerful array operations for iterating, filtering, and transforming JSON arrays. Use brackets to access elements by index, or iterate through all elements with the array iterator for processing collections efficiently.

#!/bin/bash

# Sample array
json='[{"name":"John","age":25},{"name":"Jane","age":30}]'

# Get first element
echo $json | jq '.[0]'

# Get specific field from all elements
echo $json | jq '.[].name'

# Get array length
echo $json | jq 'length'

# Filter array
echo $json | jq '.[] | select(.age > 26)'

# Map array
echo $json | jq 'map(.name)'

{
  "name": "John",
  "age": 25
}
"John"
"Jane"
2
{
  "name": "Jane",
  "age": 30
}
["John", "Jane"]

🔹 Filtering and Selection

The select function filters JSON data based on conditions. Combine it with comparison operators and logical expressions to extract exactly the data you need from complex JSON structures, making data extraction precise and efficient.

#!/bin/bash

json='[
  {"name":"Alice","age":25,"active":true},
  {"name":"Bob","age":35,"active":false},
  {"name":"Carol","age":30,"active":true}
]'

# Filter by age
echo $json | jq '.[] | select(.age > 28)'

# Filter by boolean
echo $json | jq '.[] | select(.active == true)'

# Multiple conditions
echo $json | jq '.[] | select(.age > 25 and .active == true)'

# Filter and extract field
echo $json | jq '.[] | select(.age > 25) | .name'

{
  "name": "Bob",
  "age": 35,
  "active": false
}
{
  "name": "Carol",
  "age": 30,
  "active": true
}
{
  "name": "Alice",
  "age": 25,
  "active": true
}
{
  "name": "Carol",
  "age": 30,
  "active": true
}
{
  "name": "Carol",
  "age": 30,
  "active": true
}
"Bob"
"Carol"

🔹 Transforming JSON

jq can restructure JSON data by creating new objects with selected fields, renaming keys, and combining data. This is essential for adapting API responses to your needs or preparing data for other tools and systems.

#!/bin/bash

json='{"firstName":"John","lastName":"Doe","age":30}'

# Create new object structure
echo $json | jq '{name: .firstName, years: .age}'

# Add new field
echo $json | jq '. + {country: "USA"}'

# Rename field
echo $json | jq '{name: .firstName, surname: .lastName}'

# Nested transformation
json2='{"user":{"name":"Alice","age":25}}'
echo $json2 | jq '.user | {name, age}'

{
  "name": "John",
  "years": 30
}
{
  "firstName": "John",
  "lastName": "Doe",
  "age": 30,
  "country": "USA"
}
{
  "name": "John",
  "surname": "Doe"
}
{
  "name": "Alice",
  "age": 25
}

🔹 Working with API Responses

jq is perfect for processing API responses in shell scripts. Combine it with curl to fetch data, extract specific fields, and handle complex nested JSON structures from REST APIs, making API integration in Bash scripts simple.

#!/bin/bash

# Fetch and parse API response
curl -s https://api.github.com/users/github | jq '.name'

# Extract multiple fields
curl -s https://api.github.com/users/github | \
  jq '{name: .name, followers: .followers, repos: .public_repos}'

# Save to variable
user_name=$(curl -s https://api.github.com/users/github | jq -r '.name')
echo "User: $user_name"

# Process array from API
curl -s https://api.github.com/users/github/repos | \
  jq '.[].name' | head -5

"GitHub"
{
  "name": "GitHub",
  "followers": 1234,
  "repos": 56
}
User: GitHub
"repo1"
"repo2"
"repo3"
"repo4"
"repo5"

🔹 Advanced jq Operations

Advanced jq operations unlock powerful JSON data transformation capabilities directly from the command line. Beyond simple selection, jq allows you to group_by(), sort with sort_by(), perform mathematical aggregations like sums and averages, and manipulate strings using built-in functions. You can combine filters, map values, and create custom objects to reshape complex JSON structures. This eliminates the need for auxiliary scripting languages for data-wrangling tasks, enabling efficient parsing of API responses, log files, and configuration data in a single pipeline.

#!/bin/bash

json='[{"name":"Alice","score":85},{"name":"Bob","score":92}]'

# Sort by field
echo $json | jq 'sort_by(.score)'

# Group by field
echo $json | jq 'group_by(.name)'

# Calculate sum
echo $json | jq '[.[].score] | add'

# Calculate average
echo $json | jq '[.[].score] | add / length'

# String operations
echo '{"text":"hello"}' | jq '.text | ascii_upcase'

# Conditional output
echo $json | jq '.[] | if .score > 90 then "Pass" else "Fail" end'

[
  {"name":"Alice","score":85},
  {"name":"Bob","score":92}
]
[[{"name":"Alice","score":85}],[{"name":"Bob","score":92}]]
177
88.5
"HELLO"
"Fail"
"Pass"

🔹 Common jq Use Cases

jq is indispensable for real-world scripting involving JSON data extraction and reporting. Common applications include parsing and querying configuration files like package.json or docker-compose.yml, filtering and analyzing structured log output from applications, extracting specific fields from complex API responses for monitoring, and generating formatted reports from JSON data. Its ability to handle streams makes it perfect for continuous processing in Unix pipelines, simplifying workflows that previously required custom Python or Node.js scripts.

#!/bin/bash

# Read config value
db_host=$(jq -r '.database.host' config.json)
echo "Connecting to: $db_host"

# Filter logs
jq 'select(.level == "error")' app.log

# Generate CSV
jq -r '.users[] | [.name, .email] | @csv' users.json > output.csv