What this check reads
The check takes the first 5 tagged local images, <none> entries excluded, then branches on whether trivy is installed. With trivy it runs trivy image --quiet --scanners secret --format json --timeout 120s <img> against each one and marks the image when the JSON contains "Class":"secret"; a single marked image is the FAIL and none across the sample is the only PASS this check can produce. Without trivy it greps docker history --no-trunc <img> for AWS_SECRET|PRIVATE KEY|PASSWORD=|API_KEY|SECRET_KEY and WARNs whichever way that goes, because build-command text is all that branch can see.
The trivy branch does not work as intended today. Trivy pretty-prints its JSON with a space after the colon ("Class": "secret"), so the compact pattern never matches and no image is ever marked. With trivy installed the check currently returns PASS whatever the layers contain.
When it applies
Needs the docker binary and daemon access; otherwise SKIP. It examines the first 5 tagged local images. Which branch runs depends entirely on whether trivy is installed:
- With trivy - a real secret scan of layer contents (
--scanners secret), 120-second timeout per image. This is the only branch that can PASS. - Without trivy - a much weaker fallback that greps
docker historytext forAWS_SECRET|PRIVATE KEY|PASSWORD=|API_KEY|SECRET_KEY. It only sees build-command text, never file contents, so a secret in aCOPYd file is invisible. Both outcomes of this branch are WARN, including the "looks clean" one - deliberately, because nothing was really verified.
What each result means
| Result | When | What it means |
|---|---|---|
| SKIP | Docker not installed, daemon unreachable, or no local images | Nothing was scanned. No image list was produced, so no image was checked for embedded credentials. |
| FAIL | Trivy found secrets | Trivy matched a secret pattern inside the layer contents of at least one image. This branch cannot be reached today, because the pattern never matches the JSON trivy actually prints. |
| PASS | Trivy found none | With trivy installed this is the only verdict the check can reach today, so it does not show the images are clean. Run the trivy secret scan yourself before trusting it. |
| WARN | No Trivy; secret-like strings in build history | A build command recorded in the image history contains something that looks like a credential, and anyone who can pull the image can read it. |
| WARN | No Trivy; history looks clean (weak evidence) | Nothing was really verified. With no scanner installed only build-command text was read, so a secret inside a copied file would never have been seen. |
Why it matters
Anything copied into a layer stays in the image even if a later RUN rm removes it; docker history and layer extraction recover it. Build-time ARG/ENV values are recorded in image metadata. CIS Docker 4.10 "Ensure secrets are not stored in Dockerfiles"; OWASP Docker Cheat Sheet Rule #12 (secrets); OWASP Secrets Management Cheat Sheet; NIST SP 800-190 4.1.2 "Image configuration defects / embedded secrets".
Why it fails, and when it is wrong
- The trivy match never fires (known issue). The script greps for
"Class":"secret"with no space, and trivy writes"Class": "secret". An image with a private key in a layer returns PASS. Read trivy's own output, not this verdict. - Trivy flags test fixtures, example keys and placeholders in vendor packages; check the
Title/Matchin the JSON before rotating anything. - Trivy secret scanning can be slow on large images (walks every file) → timeout → empty output → silently treated as no secrets for that image (the script does not distinguish timeout from clean). Run manually with a longer
--timeout. - The fallback history grep only sees
RUN/ENVcommand text, not file contents. - Secrets injected at run time via env vars are covered by Secrets In Env, not here.
How to fix it
Rotate the credential first
Treat it as compromised from the moment it entered a layer. Anyone who can pull the image can recover it, and neither deleting the file in a later layer nor deleting the tag undoes that.
Find where it entered the image
trivy image --scanners secret <image>
docker history --no-trunc <image> | grep -iE 'AWS_SECRET|PRIVATE KEY|PASSWORD=|API_KEY|SECRET_KEY'The first reads layer contents and names the file; the second catches a value passed as ARG or ENV.
Rebuild with a BuildKit secret mount
# syntax=docker/dockerfile:1
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm cidocker build --secret id=npmrc,src=$HOME/.npmrc -t app:1.4.3 .The secret is mounted for that one RUN and never written to a layer. Keep .env, keys and credential files out of the build context with .dockerignore, and use a multi-stage build so anything the build stage needed does not reach the final stage.
Replace every pushed copy
Push the rebuilt image under a new tag, redeploy from it, and delete the old tags from the registry. Deleting a tag does not purge the blobs straight away, which is why rotation comes first.
Verify the fix
# The real scan, across every image rather than the first five:
for i in $(docker images --format '{{.Repository}}:{{.Tag}}' | grep -v '<none>'); do
echo "== $i"; trivy image --quiet --scanners secret "$i"
done
# expected: no secret findings
# Manual inspection of build history (what the fallback sees):
docker history --no-trunc <image> | grep -iE 'AWS_SECRET|PRIVATE KEY|PASSWORD=|API_KEY|SECRET_KEY'
# And of the layers themselves (what it does not):
docker save <image> | tar -tv | headDebugging
Three different cases share this status. 'Docker not installed' is correct on a host with no Docker. 'Cannot query Docker daemon' means the audit account could not reach the daemon: check systemctl is-active docker, then docker ps as that user and sudo docker ps, and run the audit as root or with sudo if only the second works. 'No local images to scan' means the daemon held no tagged images, which is normal on a host that pulls at deploy time; check with docker images -a. It is also what you get when the audit has root but the daemon is down, because an empty answer and an empty image list look the same to the script. The Docker Inventory check covers the unreachable case in more detail.
This is the default result on a host without trivy and means nothing was verified. Treat it as unknown, not as a mild pass. Install trivy and re-run.
The fallback matched text in a build command. Confirm with the docker history command above: a literal ARG PASSWORD= or ENV API_KEY= in the history is a real exposure, since docker history is readable by anyone who can pull the image.
Treat the credential as compromised. Removing the file in a later layer does not remove it: earlier layers persist in the image and are recoverable with docker save. Rotate the secret, then rebuild without it using BuildKit secrets (RUN --mount=type=secret) or runtime injection.
Believe yourself, not the PASS. The known issue above means the trivy branch never marks an image, and on top of that only 5 tagged images are sampled. Use the loop above and read trivy's own findings.
A secret scan needs no vulnerability database, so a failed download is not the cause here. The usual one is that trivy could not read the image: it runs as the audit user, not through sudo, so on a host where only root can reach the Docker socket it cannot load local images. The other is the 120-second timeout on a large image. Run trivy image --scanners secret <image> by hand to see the real error.
Sources
- Docker: Build secrets (
--mount=type=secret) - Docker: Multi-stage builds
- Docker: .dockerignore
- Trivy: Secret scanning
- Docker: docker history
- OWASP Secrets Management Cheat Sheet
How the script reads this
Next
Re-run the Image Vulnerabilities audit after applying the fix and confirm this check moves to PASS. CtrlOps runs all 8 checks over your existing SSH connection and scores the result, so the change is visible without reading another config file.
All 8 Image Vulnerabilities fixes