Hashicorp Vault weirdness explained


A couple of days ago we where debugging an Ansible AAP problem, when a co-worker mentioned something weird in our Hashicorp Vault.
Nothing major, just weird.

He told me that sometimes and not with all entries he only could see the JSON blob of the entry and the switch to go back to normal was greyed out.

Searching Github it turned out that this is related to pull-request 4913 (https://github.com/hashicorp/vault/pull/4913), when an entry is not a string, Vault switches to the display of the JSON blob. (Thanks Chris) But, the main problem is, what is not a string??

After diving deeper it boiled down to our automated password rotation playbook. The main part below:

---
- name: Test Vault JSON toggle
  hosts: localhost
  connection: local
  become: false

  vars:
    vault_url: http://127.0.0.1:8200
    vault_path: kv
    vault_user: admin
    vault_pass: salami
    pass_chars:
      - ascii_letters
      - digits
      - punctuation

  tasks:
    - name: Generate new passwords
      ansible.builtin.set_fact:
        new_password: "{{ lookup('ansible.builtin.password',
          '/dev/null',
          length=30, chars=pass_chars) }}"

    - name: Read the latest version for oldpass
      community.hashi_vault.vault_kv2_get:
        auth_method: userpass
        url: "{{ vault_url }}"
        username: "{{ vault_user }}"
        password: "{{ vault_pass }}"
        validate_certs: false
        engine_mount_point: "{{ vault_path }}"
        path: "{{ '/password' }}"
      register: oldpass
      ignore_errors: true

    - name: Show data
      debug:
        msg:
          - "{{ oldpass.data.data | default('Unknown') }}"

    - name: Combine existing passwords with new ones
      ansible.builtin.set_fact:
        new_passwords: "{{ (oldpass.data.data | default({})) | combine({'root': new_password}) }}"

    - name: Show data
      debug:
        msg:
          - "{{ new_passwords }}"

    - name: Store passwords in vault
      community.hashi_vault.vault_write:
        auth_method: userpass
        url: "{{ vault_url }}"
        username: "{{ vault_user }}"
        password: "{{ vault_pass }}"
        validate_certs: false
        path: "{{ vault_path + '/data/password' }}"
        data:
          data: "{{ new_passwords }}"

When having a look at the Vault web-interface it just shows the normal way of displaying entries.

image::/images/vault_normal.jpg[Normal vault,title="Normal Vault display"]

If I run that playbook multiple times, it sometimes shows the entry as a JSON blob, like this:

JSON vault
Figure 1. JSON Vault display

When looking further I notices that every time a JSON blob was displayed there was a { in the password, and as that could be part of a JSON data set, that could be the culprit. Adding all kind of special characters to the string, it turns out that only the { messes things up. It’s even so that when you change the { into something else, Vault automatically displays the entry the normal way again.

Problem solved, this was a nice witch hunt.

Only thing left to do is change the upper part of the playbook into

    pass_chars:
      - ascii_letters
      - digits
      - ".,/:-_<>~!@#$%^&*()[]"

And be done with it.

vault 

See also