Summary
ops leaking secrets if subprocess.CalledProcessError happens with a secret-* CLI command
The issue here is that we pass the secret content as one of the args via CLI. This issue may affect any of our charms that are using: Juju (>=3.0), Juju secrets and not correctly capturing and processing subprocess.CalledProcessError.
There are two points that may log this command, in different files:
First, if there is an error during a secret handling, there will be a subprocess.CalledProcessError, which will contain the CLI comand + all its args. This is going to be logged in any logging level. This exception, if not caught by the charm, will bubble up to the /var/log/juju/ logs and syslog journal. Now, on Ubuntu 22.04, these logs are protected with:
$ juju ssh -m controller 0 -- ls -la /var/log/juju/
total 224
drwxr-xr-x 2 syslog adm 4096 Jul 14 10:59 .
drwxrwxr-x 9 root syslog 4096 Jul 14 10:58 ..
-rw-r----- 1 syslog adm 20124 Jul 14 11:10 audit.log
-rw-r----- 1 syslog adm 110432 Jul 14 11:10 logsink.log
-rw-r----- 1 syslog adm 80783 Jul 14 11:06 machine-0.log
-rw-r----- 1 syslog adm 766 Jul 14 11:10 machine-lock.log
-rw-r--r-- 1 root root 0 Jul 14 10:59 slow-query.log
-rw-r----- 1 syslog adm 3732 Jul 14 11:10 unit-controller-0.log
Second, certain audit setups may log terminal commands, which would result in this command being logged with its secrets. It is unknown if this is done on ubuntu security benchmarks, such as CIS hardening.
Keep in mind these logs may be copied or even backed up. Which exposes it to more services in the user's environment (e.g. CI runs in GH - although these are dummy password generated per test only).
Passing secrets straight via CLI is not advised. Here are some ways out:
- Redacting: which commands and which args represent secrets are known, so they can be redacted. It would also mean capturing a
subprocess.CalledProcessError, redacting its content and reissuing the same type of exception; this will not cover the caseauditdis set to log CLI commands, if that is a risk - Temp files: secret-add, for example, can use a secret file instead, as can be seen here. However, if ops uses a file, ops will need to be sure to correctly remove it later
- stdin: not sure it is accepted by secret-* commands, but generally, secrets are not shown on CLI whilst typing them; auditd may not capture that stdin
Severity Rationale
This is a CWE-532. Potentially, these secrets can lead to privilege escalation but Ubuntu default is to have logs only accessible to adm group users.
Marking this issue as "Moderate", as this report is not presenting a clear way on how to get access to the logs themselves: either getting local access to an adm group user (e.g. ubuntu) or recovering logs stored on a 3rd party service.
Details
2024-07-12T14:27:58.0175389Z unit-opensearch-3: 14:27:53 ERROR unit.opensearch/3.juju-log certificates:11: Uncaught exception while in charm code:
2024-07-12T14:27:58.0175524Z Traceback (most recent call last):
2024-07-12T14:27:58.0175957Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/model.py", line 3180, in _run
2024-07-12T14:27:58.0176165Z result = subprocess.run(args, **kwargs) # type: ignore
2024-07-12T14:27:58.0176381Z File "/usr/lib/python3.10/subprocess.py", line 526, in run
2024-07-12T14:27:58.0176561Z raise CalledProcessError(retcode, process.args,
2024-07-12T14:27:58.0177749Z subprocess.CalledProcessError: Command '('/var/lib/juju/tools/unit-opensearch-3/secret-add', '--label', 'opensearch:app:app-admin', '--owner', 'application', 'keystore-password-ca=aUE...')' returned non-zero exit status 1.
2024-07-12T14:27:58.0177765Z
2024-07-12T14:27:58.0178023Z The above exception was the direct cause of the following exception:
2024-07-12T14:27:58.0178038Z
2024-07-12T14:27:58.0178168Z Traceback (most recent call last):
2024-07-12T14:27:58.0178588Z File "/var/lib/juju/agents/unit-opensearch-3/charm/./src/charm.py", line 213, in <module>
2024-07-12T14:27:58.0178724Z main(OpenSearchOperatorCharm)
2024-07-12T14:27:58.0179129Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/main.py", line 548, in main
2024-07-12T14:27:58.0179237Z manager.run()
2024-07-12T14:27:58.0179640Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/main.py", line 527, in run
2024-07-12T14:27:58.0179745Z self._emit()
2024-07-12T14:27:58.0180150Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/main.py", line 516, in _emit
2024-07-12T14:27:58.0180359Z _emit_charm_event(self.charm, self.dispatcher.event_name)
2024-07-12T14:27:58.0180840Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/main.py", line 147, in _emit_charm_event
2024-07-12T14:27:58.0180974Z event_to_emit.emit(*args, **kwargs)
2024-07-12T14:27:58.0181537Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 348, in emit
2024-07-12T14:27:58.0181656Z framework._emit(event)
2024-07-12T14:27:58.0182091Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 860, in _emit
2024-07-12T14:27:58.0182211Z self._reemit(event_path)
2024-07-12T14:27:58.0182659Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 950, in _reemit
2024-07-12T14:27:58.0182770Z custom_handler(event)
2024-07-12T14:27:58.0183492Z File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/tls_certificates_interface/v3/tls_certificates.py", line 1811, in _on_relation_changed
2024-07-12T14:27:58.0183743Z self.on.certificate_available.emit(
2024-07-12T14:27:58.0184173Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 348, in emit
2024-07-12T14:27:58.0184291Z framework._emit(event)
2024-07-12T14:27:58.0184717Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 860, in _emit
2024-07-12T14:27:58.0184887Z self._reemit(event_path)
2024-07-12T14:27:58.0186171Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 950, in _reemit
2024-07-12T14:27:58.0186306Z custom_handler(event)
2024-07-12T14:27:58.0187043Z File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_tls.py", line 209, in _on_certificate_available
2024-07-12T14:27:58.0187320Z self.store_new_ca(self.charm.secrets.get_object(scope, cert_type.val))
2024-07-12T14:27:58.0187942Z File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_tls.py", line 444, in store_new_ca
2024-07-12T14:27:58.0188242Z self._create_keystore_pwd_if_not_exists(Scope.APP, CertType.APP_ADMIN, "ca")
2024-07-12T14:27:58.0188981Z File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_tls.py", line 432, in _create_keystore_pwd_if_not_exists
2024-07-12T14:27:58.0189119Z self.charm.secrets.put_object(
2024-07-12T14:27:58.0189738Z File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_secrets.py", line 359, in put_object
2024-07-12T14:27:58.0189944Z self._add_or_update_juju_secret(scope, key, value, merge)
2024-07-12T14:27:58.0190652Z File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_secrets.py", line 272, in _add_or_update_juju_secret
2024-07-12T14:27:58.0190815Z return self._add_juju_secret(scope, key, value)
2024-07-12T14:27:58.0191462Z File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_secrets.py", line 228, in _add_juju_secret
2024-07-12T14:27:58.0191660Z secret = scope_obj.add_secret(safe_value, label=label)
2024-07-12T14:27:58.0192110Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/model.py", line 477, in add_secret
2024-07-12T14:27:58.0192241Z id = self._backend.secret_add(
2024-07-12T14:27:58.0192686Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/model.py", line 3624, in secret_add
2024-07-12T14:27:58.0192948Z result = self._run('secret-add', *args, return_output=True)
2024-07-12T14:27:58.0193356Z File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/model.py", line 3182, in _run
2024-07-12T14:27:58.0193489Z raise ModelError(e.stderr) from e
2024-07-12T14:27:58.0193685Z ops.model.ModelError: ERROR this unit is not the leader
2024-07-12T14:27:58.0193692Z
From CI: https://github.com/canonical/opensearch-operator/actions/runs/9908987369/job/27376377521?pr=364
PoC
- Deploy anything with juju
- Run a dummy secret-add call that will fail
- See the uncaught subprocess error
Impact
Juju secrets are generally composed of private keys, passwords, etc; generally valuable credentials that, if leaked, will likely allow an attacker to get privileged access to its target or other targets in the environment.
CVE-2024-41129 has a CVSS score of 4.4 (Medium). The vector is requires local access, high privileges required, and no user interaction. A CVSS score reflects the worst-case severity of the vulnerability, not your specific exposure. Whether this affects your application depends on whether the vulnerable code is present and reachable in your environment. A fixed version is available (2.15.0); upgrading removes the vulnerable code path.
Affected versions
Security releases
Kodem intelligence
Severity tells you how bad this could be in the worst case. It does not tell you whether you are exposed. Exploitability and impact are functions of runtime truth: whether the vulnerable code is present, reachable, and actually executes in your application. A vulnerable package can sit in your dependency tree and never run.
Kodem, an Intelligent Application Security platform, uses runtime intelligence to reveal which vulnerabilities actually execute in production, so teams prioritize the ones that genuinely matter. Kodem's runtime-powered SCA identifies whether this CVE is reachable in your applications.
Already deployed Kodem?
See it in your environmentNew to Kodem? Get a demo →Remediation advice
Kodem Kai can prioritize this vulnerability in your dependency tree and generate a fix recommendation.
Frequently Asked Questions
- What is CVE-2024-41129? CVE-2024-41129 is a medium-severity security vulnerability in ops (pip), affecting versions >= 2.0.0, < 2.15.0. It is fixed in 2.15.0.
- How severe is CVE-2024-41129? CVE-2024-41129 has a CVSS score of 4.4 (Medium). This score reflects the worst-case severity of the vulnerability, not your specific exposure. Whether it represents real risk in your environment depends on whether the vulnerable code is present and reachable.
- Which versions of ops are affected by CVE-2024-41129? ops (pip) versions >= 2.0.0, < 2.15.0 is affected.
- Is there a fix for CVE-2024-41129? Yes. CVE-2024-41129 is fixed in 2.15.0. Upgrade to this version or later.
- Is CVE-2024-41129 exploitable, and should I be worried? Whether CVE-2024-41129 is exploitable in your environment depends on whether the vulnerable code is present and reachable. A CVSS score is a worst-case rating; it does not account for your specific deployment, configuration, or usage patterns. Kodem, an Intelligent Application Security platform, uses runtime intelligence to show which vulnerabilities actually execute in production, so you can focus on the ones that represent real risk. Get a demo
- What actually determines whether CVE-2024-41129 is exploitable, and how bad it is? Exploitability and impact are not fixed properties of a CVE. They depend on runtime truth: whether the vulnerable code is present, reachable, and actually executes in your application. A high CVSS score on a dependency that never runs is not the same as real risk. Kodem, an Intelligent Application Security platform, uses runtime intelligence to reveal which vulnerabilities actually execute in production, so teams prioritize the ones that genuinely matter.
- How do I fix CVE-2024-41129? Upgrade
opsto 2.15.0 or later.