Using jq to Append a New Element to an Existing JSON Array

The code I’m using adds the new object at the bottom, as shown in the desired output. To achieve this, you can concatenate arrays by adding the new object to an array and using the concat() method to join it with the rest of the array. Here’s an example implementation in JavaScript: https://jqplay.org/s/XIsoZ4GvO. Another solution for JavaScript arrays is to use the push() method.

Question:

I am attempting to add a new element to a json file’s array using the

jq``add

command, but it is not functioning properly.


report-2017-01-07.json

file:

{  
   "report": "1.0",
   "data": {  
      "date": "2010-01-07",
      "messages": [  
         {  
            "date": "2010-01-07T19:58:42.949Z",
            "xml": "xml_samplesheet_2017_01_07_run_09.xml",
            "status": "OK",
            "message": "metadata loaded into iRODS successfully"
         },
         {  
            "date": "2010-01-07T20:22:46.949Z",
            "xml": "xml_samplesheet_2017_01_07_run_09.xml",
            "status": "NOK",
            "message": "metadata duplicated into iRODS"
         },
         {  
            "date": "2010-01-07T22:11:55.949Z",
            "xml": "xml_samplesheet_2017_01_07_run_09.xml",
            "status": "NOK",
            "message": "metadata was not validated by XSD schema"
         }
      ]
   }
}

I am using this command:

$ cat report-2017-01-07.json 
| jq -s '.data.messages {"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}'
jq: error: syntax error, unexpected '{', expecting $end (Unix shell quoting issues?) at , line 1:
.data.messages {"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}               
jq: 1 compile error

This is the desired appearance of the output.

{
    "report": "1.0",
    "data": {
        "date": "2010-01-07",
        "messages": [{
            "date": "2010-01-07T19:58:42.949Z",
            "xml": "xml_samplesheet_2017_01_07_run_09.xml",
            "status": "OK",
            "message": "metadata loaded into iRODS successfully"
        }, {
            "date": "2010-01-07T20:22:46.949Z",
            "xml": "xml_samplesheet_2017_01_07_run_09.xml",
            "status": "NOK",
            "message": "metadata duplicated into iRODS"
        }, {
            "date": "2010-01-07T22:11:55.949Z",
            "xml": "xml_samplesheet_2017_01_07_run_09.xml",
            "status": "NOK",
            "message": "metadata was not validated by XSD schema"
        }, {
            "date": "2010-01-07T19:55:99.999Z",
            "xml": "xml_samplesheet_2017_01_07_run_09.xml",
            "status": "OKKKKKKK",
            "message": "metadata loaded into iRODS successfullyyyyy"
        }]
    }
}



Solution 1:

By utilizing the filter, a fresh array element can be included through the

|= .+

component. Employing the filter with

jq

is also a feasible approach.

jq '.data.messages[3] |= . + {"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}' inputJson

Instead of relying on a fixed length value like in

3

, use

. | length

to dynamically add new elements to the array. This function returns the length, which can then be used as the next array index.

jq '.data.messages[.data.messages| length] |= . + {"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}' inputJson

Alternatively, following Peak’s recommendation in the comments, utilizing solely the operator

+=

.

jq '.data.messages += [{"date": "2010-01-07T19:55:99.999Z", "xml": "xml_samplesheet_2017_01_07_run_09.xml", "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}]'

which produces the output you need:

{
  "report": "1.0",
  "data": {
    "date": "2010-01-07",
    "messages": [
      {
        "date": "2010-01-07T19:58:42.949Z",
        "xml": "xml_samplesheet_2017_01_07_run_09.xml",
        "status": "OK",
        "message": "metadata loaded into iRODS successfully"
      },
      {
        "date": "2010-01-07T20:22:46.949Z",
        "xml": "xml_samplesheet_2017_01_07_run_09.xml",
        "status": "NOK",
        "message": "metadata duplicated into iRODS"
      },
      {
        "date": "2010-01-07T22:11:55.949Z",
        "xml": "xml_samplesheet_2017_01_07_run_09.xml",
        "status": "NOK",
        "message": "metadata was not validated by XSD schema"
      },
      {
        "date": "2010-01-07T19:55:99.999Z",
        "xml": "xml_samplesheet_2017_01_07_run_09.xml",
        "status": "OKKK",
        "message": "metadata loaded into iRODS successfullyyyyy"
      }
    ]
  }
}

Optimize your

jq-filter

by dry-running it using jq-play, and make any desired adjustments.


Solution 2:


Instead of opting for

|=

, it is suggested to go for

+=

.

.data.messages += [{"date": "2010-01-07T19:55:99.999Z",
   "xml": "xml_samplesheet_2017_01_07_run_09.xml",
   "status": "OKKK", "message": "metadata loaded into iRODS successfullyyyyy"}]

Prepend

Alternatively, if your intention is to prepend the
JSON object
to the array, as suggested by @NicHuang, you may utilize the following format:

 .data.messages |= [ _ ] + .


Solution 3:

Summary: “. +” is your saviour

Details:

To add an entry to a list, it is recommended to use the append function with [list1] and [list2], rather than combining [list] with data.

$ echo '[ "data1" ]' | jq '. + [ "data2" ]'
[
  "data1",
  "data2"
]
$ echo '[ {"key1": "value1"} ]' | jq '. + [{"key2": "value2"}]'
[
  {
    "key1": "value1"
  },
  {
    "key2": "value2"
  }
]

To append a new key and value to a dictionary.

$ echo '{"key1": "value1"}' | jq '. + {"key2": "value2"}'
{
  "key1": "value1",
  "key2": "value2"
}

References:

Access the contents of the given link on Github, which is a gist created by a user named joar with a unique identifier of 776b7d176196592ed5d8.

Frequently Asked Questions