Skip to content

Module 07: Reconnaissance and OSINT

← Module 06: Broken Access Control | Topic Home | Next: Exploitation Basics →

Status Difficulty Time

Reconnaissance determines which 10% of your attack surface is actually attackable — doing it well means you spend time on the right targets, not the hardened ones.

[!NOTE] Stub Module — Full theory content will be expanded in a future update.

Table of Contents

  1. Overview
  2. Prerequisites
  3. Objectives
  4. Theory
  5. Key Concepts
  6. Examples
  7. Common Pitfalls
  8. Cross-Links
  9. Summary

Overview

Reconnaissance is the phase where more pentests succeed or fail than anywhere else. A tester who rushes past reconnaissance and jumps to exploitation will miss the low-hanging fruit that a careful reconnaissance phase would have revealed: the forgotten subdomain running an old application, the admin panel exposed to the internet, the server banner revealing an unpatched version.

Reconnaissance splits into passive (OSINT) — gathering information without touching the target — and active — directly interacting with the target. This module covers both, with emphasis on OSINT techniques that are invisible to the target and active techniques (port scanning, subdomain enumeration) that are authorized by the engagement scope.

[!IMPORTANT] All active reconnaissance (port scanning, directory fuzzing, subdomain enumeration) requires explicit authorization. Passive OSINT uses only publicly available information and does not require touching the target's systems.

Prerequisites

  • [[pentesting-security/modules/02_networking-for-security]] — nmap, DNS, HTTP fundamentals
  • [[pentesting-security/modules/01_introduction]] — Authorization and scope requirements

Objectives

By the end of this module, you will be able to:

  1. Distinguish passive OSINT from active reconnaissance and know when each is appropriate
  2. Use Google dorking, Shodan, and certificate transparency logs for passive recon
  3. Perform subdomain enumeration using gobuster and passive CT log sources
  4. Use theHarvester and similar tools for organizational OSINT
  5. Conduct active recon (port scan, directory bruteforce) against an authorized target
  6. Build a structured reconnaissance report documenting your findings

Theory

Passive vs. Active Reconnaissance

Passive reconnaissance (OSINT): Gathering information from publicly available sources without sending any traffic to the target. The target cannot detect passive recon.

Sources for passive recon: - DNS records: A, MX, TXT, CNAME records (via DNS lookup tools, no querying target servers) - WHOIS: Domain registration data (owner, registrar, dates) — often partially anonymized - Certificate Transparency (CT) logs: Every SSL/TLS certificate issued is logged publicly at crt.sh. Querying CT logs reveals subdomains without touching the target. - Shodan: Internet-wide port scanning data indexed by IP address. Shows open ports, service banners, and certificates for any IP — without you scanning the target at all. - Google dorking: Advanced Google search operators to find exposed data. - LinkedIn/social media: Employee names, job titles, technology stack (from job postings). - GitHub: Source code leaks, API keys, internal hostnames, credentials in commit history. - Archive.org Wayback Machine: Historical versions of websites and pages.

Active reconnaissance: Directly interacting with the target. Detectable; requires authorization.

# Passive: Certificate Transparency logs (no traffic to target)
# View at https://crt.sh/?q=%.example.com
# Via API:
curl "https://crt.sh/?q=%.example.com&output=json" | python3 -m json.tool | grep "name_value"

# Passive: Shodan search (no traffic to target)
# In browser: https://www.shodan.io/search?query=hostname%3Aexample.com
# Via CLI (requires free Shodan account):
shodan search "hostname:example.com"
shodan host 203.0.113.100  # Lookup specific IP

# Active: Subdomain enumeration (requires authorization)
gobuster dns -d example.com \
  -w /usr/share/wordlists/seclists/Discovery/DNS/subdomains-top1million-5000.txt \
  -t 20

Google Dorking

Google's advanced operators can find information that is technically public but not meant to be easily found. Used ethically in authorized pentests for reconnaissance.

# Common Google Dorks:

# Find login pages on a domain
site:example.com inurl:login
site:example.com inurl:admin

# Find exposed configuration files
site:example.com ext:env
site:example.com ext:xml inurl:config
site:example.com filetype:sql

# Find all subdomains indexed by Google
site:*.example.com

# Find specific error messages that reveal technology
site:example.com "SQL syntax error"
site:example.com "Fatal error"

# Find exposed documents
site:example.com filetype:pdf "confidential"
site:example.com filetype:xls

# Find exposed cameras, devices (ethical recon on own infrastructure only)
intitle:"Index of" inurl:password.txt

theHarvester — Organizational OSINT

# Gather emails, subdomains, IPs from public sources
theHarvester -d example.com -b google,bing,linkedin,twitter,github

# -d: target domain
# -b: data sources (use -b all for all sources — slower)
# Output: email addresses, discovered subdomains, IPs

# Output to file
theHarvester -d example.com -b all -f ~/recon/harvester_output

Directory and File Discovery (Active)

# gobuster directory mode — find hidden paths
gobuster dir \
  -u http://target.example.com \
  -w /usr/share/wordlists/dirb/common.txt \
  -x php,html,txt,bak,zip \
  -t 20 \
  -o ~/recon/gobuster_dirs.txt

# ffuf — faster, more flexible alternative
ffuf \
  -u http://target.example.com/FUZZ \
  -w /usr/share/wordlists/seclists/Discovery/Web-Content/Web-Extensions.fuzz.txt \
  -fs 0  # Filter by response size (0 = empty responses)

Shodan for Infrastructure Discovery

import shodan  # pip install shodan

# Shodan API (free tier: 1 query/second, limited results)
api = shodan.Shodan("YOUR_API_KEY")  # From https://account.shodan.io/

# Search by hostname
results = api.search("hostname:example.com")
for result in results['matches']:
    print(f"IP: {result['ip_str']}")
    print(f"Port: {result['port']}")
    print(f"Banner: {result.get('data', '')[:200]}")
    print(f"CVEs: {result.get('vulns', 'None')}")
    print("---")

# Search for specific vulnerable versions (e.g., for authorized target research)
results = api.search("product:Apache version:2.4.49")

Key Concepts

Attack Surface — The complete set of entry points that a tester should investigate. Recon is the phase that maps the attack surface comprehensively before testing begins.

Subdomain Takeover — A vulnerability where a subdomain points to a service (S3, GitHub Pages, Heroku, etc.) that no longer exists. An attacker can register the dangling service and serve content on the subdomain. Found via passive DNS/CT log enumeration.

Google Dork — A Google search using advanced operators (site:, inurl:, filetype:, intitle:) to find specific information about a target from public sources.

Information Disclosure — Any unintended exposure of system information: server banners, error messages with stack traces, source code comments, backup files. Reconnaissance specifically hunts for information disclosure because it reveals technology stack, version information, and potentially credentials.

Examples

Example 1: Complete Passive Recon Workflow

# Passive recon for example.com (no traffic to target)

# 1. DNS records
dig any example.com
host -t mx example.com
host -t txt example.com  # Look for SPF, DKIM, verification tokens

# 2. WHOIS
whois example.com | grep -E "Registrar|Creation|Expiry|Name Server"

# 3. Certificate Transparency
curl -s "https://crt.sh/?q=%.example.com&output=json" | \
  python3 -c "import json,sys; [print(c['name_value']) for c in json.load(sys.stdin)]" | \
  sort -u

# 4. Google dorks (run manually in browser)
# site:example.com
# site:*.example.com
# site:example.com filetype:pdf
# site:example.com inurl:admin

# 5. theHarvester
theHarvester -d example.com -b google,bing -l 100

# Compile results into a structured recon report

Common Pitfalls

Pitfall 1: Starting active recon before passive recon. Passive recon is free intelligence that costs you nothing and leaves no trace. Always exhaust passive sources first.

Pitfall 2: Missing subdomain takeover vulnerabilities. CT log enumeration plus CNAME resolution checking can find subdomains that point to dangling services. This is high-impact and often missed by automated scanners.

Pitfall 3: Ignoring GitHub. Source code repositories, including open-source dependencies, often contain hardcoded API keys, internal hostnames, database credentials, and other secrets. GitHub search and tools like TruffleHog or gitleaks can find these.

  • [[pentesting-security/modules/02_networking-for-security]] — Port scanning (nmap) used in active recon
  • [[pentesting-security/modules/08_exploitation-basics]] — Module 08 starts where recon ends
  • [[pentesting-security/modules/11_bug-bounty-program]] — Bug bounty success starts with thorough recon

Summary

  • Passive OSINT gathers information from public sources without touching the target; always do this first
  • CT logs, Shodan, Google dorking, and theHarvester are the primary passive recon tools
  • Active recon (port scans, directory fuzzing, subdomain brute force) requires authorization
  • A comprehensive recon report maps the entire attack surface before any exploitation attempts
  • Subdomain takeover is a high-impact vulnerability found through passive DNS/CT enumeration
  • GitHub and other code repositories are often overlooked but can yield critical secrets