Multi tool use

jq shell script: aggregate iteration content into json body

I'm creating a shell script.

I need to create a curl json body according the a key-value list content. This content is splitted by way of a awk which generate a two column table:

key-value

awk

KVS_VARIABLES=$(awk -F= '!($1 && $2 && NF==2) { print "File failed validation on line " NR | "cat 1>&2"; next } { print $1, $2 }' $f)


Example output:

VAR1 VAL1
VAR2 VAL2
VAR3 VAL3


So, this table is iterated on a while iteration and each key and value are splitted:

key

value

echo "$KVS_VARIABLES" | while read -r kv
do
key=$(echo $kv | awk '{print $1}')
value=$(echo $kv | awk '{print $2}')
done


So, I need some way to aggregate this content into a json document in order to send it out using curl:

curl

curl -k
-X PUT
-d @-
-H "Authorization: Bearer $TOKEN"
-H "Accept: application/json"
-H "Content-Type: application/json"
"$SERVER_URL/api/v1/namespaces/$NAMESPACE/secrets/$SECRET_ID" <<-EOF
{
"kind": "Secret",
"apiVersion": "v1",
"metadata": {
"name": "$SECRET_ID"
},
"stringData": {
"$key": "$value" <<<<<<<<<<<<<(1)>>>>>>>>>>>>>>
}
}
EOF


So, on <<<<<<<<<<<<<(1)>>>>>>>>>>>>>> I need to aggregate each key and value propagation.`

<<<<<<<<<<<<<(1)>>>>>>>>>>>>>>

key

value

So, in this case I'd need to generate:

"VAR1": "VAL1",
"VAR2": "VAL2",
"VAR3": "VAL3"


and then insert it inside "stringData":

{
"kind": "Secret",
"apiVersion": "v1",
"metadata": {
"name": "$SECRET_ID"
},
"stringData": {
<<<<<<<<<<<<<(1)>>>>>>>>>>>>>>
}
}


So, after all:

{
"kind": "Secret",
"apiVersion": "v1",
"metadata": {
"name": "$SECRET_ID"
},
"stringData": {
"VAR1": "VAL1",
"VAR2": "VAL2",
"VAR3": "VAL3"
}
}


jq is installed.

jq

Any ideas?

1 Answer
1

You don't need an awk statement inside the while loop, but just read the key value pairs inside the read command itself.

awk

read

Also storing awk output in a variable and later parsing is an anti-pattern. You could use the process substitution feature provided by the shell, the < <() part will slurp the output of a command as if it were appearing on a file (or) use the here-strings

awk

< <()

json=$(cat <<-EOF
{
"kind": "Secret",
"apiVersion": "v1",
"metadata": {
"name": "$SECRET_ID"
},
"stringData": {
}
}
EOF
)

while read -r key value; do
json=$(echo "$json" | jq ".stringData += { "$key" : "$value" }")
done< <(awk -F= '!($1 && $2 && NF==2) { print "File failed validation on line " NR | "cat 1>&2"; next } { print $1, $2 }' $f)


You could now use the variable "$json" in the curl as

"$json"

curl

curl -k
-X PUT
-d @-
-H "Authorization: Bearer $TOKEN"
-H "Accept: application/json"
-H "Content-Type: application/json"
"$SERVER_URL/api/v1/namespaces/$NAMESPACE/secrets/$SECRET_ID" <<<"$json"


done< <(awk -F= '!($1 && $...

Syntax error: redirection unexpected

#!/usr/bin/env bash

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.