Discover directories and files with controlled response filtering
Needs: Correct base URL/session and a measured nonexistent-path response; bounded relevant wordlist
Shell commands
# Baseline
gobuster dir -u http://TARGET:PORT/ -w /usr/share/wordlists/dirb/common.txt -t 10 -o gobuster-root.txt
# Variant — technology-aware extensions
gobuster dir -u http://TARGET:PORT/ -w WORDLIST -x php,txt,bak,old,zip -t 10 -o gobuster-ext.txt
# Variant — authenticated discovery
gobuster dir -u http://TARGET:PORT/ -w WORDLIST -c 'SESSION=VALUE' -H 'Authorization: Bearer TOKEN' -t 10 -o gobuster-auth.txt
# Variant — filter a confirmed wildcard body length
gobuster dir -u http://TARGET:PORT/ -w WORDLIST --exclude-length NOISE_LENGTH -t 10 -o gobuster-filtered.txt
Decisive output: manually verified routes including useful
200, redirect,401,403and405responses.
Use only the cookie or header that applies. An expired session can turn every result into the same login redirect.
Verify → Open candidates and record their actual behavior; status/length alone is only a selection clue.
If it fails → Aggressive filters can hide genuine routes and rate limiting can fabricate differences; validate the baseline and reduce the request rate. More detail ↓
Explanation & examples
Discover directories and files with controlled response filtering
- USE WHEN
- An application has unlinked paths, backups or administration endpoints
- STARTING ACCESS
- A reachable HTTP application and recorded anonymous or authenticated request state
- RESULT
- Verified candidate routes and files
Recognize the clue
Establish the application's missing-path response, then run a bounded content search with the existing session/host context.
curl -sk -D - http://TARGET:PORT/KNOWN_MISSING_PATH
Missing control: status=200, body=DEFAULT_ERROR
Candidate /EXPORT_PATH/: status=200, body=EXPORT_FORMMeans: The candidate's actual function differs from the soft-404 control despite equal status codes.
Confirm: Request a known-missing path under the same host/session and compare status, body and length. Open each distinct Gobuster/ffuf result; a uniform login page or soft-404 is a filter problem, not a directory finding.
- After validation
- Independent retrieval confirms an actual function: add its request to the behavior map.
- Ruled out in this context
- The candidate matches the wildcard error body: remove it from the useful queue.
- Still inconclusive
- Rate limiting or session expiry distorts response size; restore stable controls before filtering.
Troubleshooting: Aggressive filters can hide genuine routes and rate limiting can fabricate differences; validate the baseline and reduce the request rate.
Request a random nonexistent path first and record its status, length and body marker.
Decisive output: manually verified routes including useful
200, redirect,401,403and405responses.
Variant — technology-aware extensions
Choose extensions from observed stack evidence; do not spray every extension.
Variant — authenticated discovery
Use only the cookie or header that applies. An expired session can turn every result into the same login redirect.
Variant — filter a confirmed wildcard body length
Filtering without first confirming the noise response can hide a real path.
Why it works
Discovery enumerates names; wildcard responses, redirects and soft-404 pages can make every guess appear valid. Confirm candidates through independent retrieval and behavior.
Worked example · hypothetical
Observation: every path returns 200. Test: compare the missing-path body/length before filtering. Illustrative output: one candidate has a distinct export response. Interpretation: inspect that function manually.
New identity, privilege, host or route? Re-enumerate → All topic tasks →