HTTPXRay 2.0 CLI Command Reference
===================================

Script name
-----------
cli.py

Default database
----------------
httpxray.db

By default, the database lives beside cli.py or beside the compiled executable.

Basic invocation
----------------
python cli.py
python cli.py shell
python cli.py --help

Running without a subcommand starts the interactive HTTPXRay shell.

Global syntax
-------------
python cli.py [--db DATABASE] <command> [options]

Global option
-------------
--db DATABASE
    Use a specific SQLite database instead of the default httpxray.db.

Examples:
    python cli.py --db scans/site-a.db stats
    python cli.py --db archive.db list --limit 50


DIRECT COMMAND MODE
===================

1. crawl
--------
Crawl a target and store discovered link intelligence in SQLite.

Syntax:
    python cli.py crawl <seed> [--external] [--workers N] [--depth N] [--clear-first]

Arguments:
    <seed>
        Required target URL.
        Must begin with http:// or https://

Options:
    --external
        Follow external links as well as internal links.

    --workers N
        Number of crawler worker threads.
        Default: 10

    --depth N
        Maximum crawl depth.
        Default: 25

    --clear-first
        Clear the active HTTPXRay database before starting the crawl.

Examples:
    python cli.py crawl https://example.com

    python cli.py crawl https://example.com --workers 20

    python cli.py crawl https://example.com --depth 50

    python cli.py crawl https://example.com --external

    python cli.py crawl https://example.com --external --workers 20 --depth 50

    python cli.py crawl https://example.com --clear-first

Recommended clean-crawl workflow:
    python cli.py db-export previous-crawl.db
    python cli.py crawl https://example.com --clear-first


2. stats
--------
Display statistics from the active database.

Syntax:
    python cli.py stats

Reports:
    Total records
    Visited records
    Internal links
    External links
    Successful visits
    Errors

Example:
    python cli.py stats


3. list
-------
List stored records in the terminal.

Syntax:
    python cli.py list [--filter TEXT] [--limit N]

Options:
    --filter TEXT
        Filter stored records.

    --limit N
        Maximum number of records to print.
        Default: 100

Examples:
    python cli.py list

    python cli.py list --limit 250

    python cli.py list --filter wordpress

    python cli.py list --filter ".pdf" --limit 50


4. export
---------
Export stored records.

Supported formats:
    csv
    json
    txt
    html
    pdf

Syntax:
    python cli.py export <format> <output> [--filter TEXT]

Arguments:
    <format>
        csv | json | txt | html | pdf

    <output>
        Destination filename.

Option:
    --filter TEXT
        Export only matching records.

Examples:
    python cli.py export csv links.csv

    python cli.py export json links.json

    python cli.py export txt links.txt

    python cli.py export html report.html

    python cli.py export pdf report.pdf

    python cli.py export html wordpress.html --filter wordpress

    python cli.py export csv errors.csv --filter error

Notes:
    Export works from the active SQLite database.
    A filter affects only the exported record set.
    It does not modify or delete database contents.


5. db-export
------------
Create a complete SQLite backup/archive of the active HTTPXRay database.

Syntax:
    python cli.py db-export <output>

Example:
    python cli.py db-export example-com-2026-08-11.db

Another database may also be selected globally:
    python cli.py --db httpxray.db db-export archive.db

Use this before clearing a crawl when you want to preserve it.


6. db-import
------------
Replace the active HTTPXRay database with an existing compatible HTTPXRay archive.

Syntax:
    python cli.py db-import <input> [--yes]

Arguments:
    <input>
        Existing HTTPXRay SQLite database.

Option:
    --yes
        Skip the interactive replacement confirmation.

Examples:
    python cli.py db-import archived-crawl.db

    python cli.py db-import archived-crawl.db --yes

    python cli.py --db working.db db-import archived-crawl.db --yes

Important:
    Database import replaces the active database contents.
    Export the current database first if the crawl must be preserved.


7. clear
--------
Clear all stored link records from the active database while keeping the database/schema ready for another crawl.

Syntax:
    python cli.py clear [--yes]

Option:
    --yes
        Skip confirmation.

Examples:
    python cli.py clear

    python cli.py clear --yes

Equivalent clean-crawl shortcut:
    python cli.py crawl https://example.com --clear-first


8. view
-------
Generate a standalone HTML view of stored records and open it in the default browser.

Syntax:
    python cli.py view [--filter TEXT] [--output FILE]

Options:
    --filter TEXT
        Include only matching records.

    --output FILE
        HTML file to create.
        Default: links_view.html beside the application.

Examples:
    python cli.py view

    python cli.py view --filter wordpress

    python cli.py view --output site-report.html

    python cli.py view --filter ".js" --output javascript-links.html


9. shell
--------
Launch the interactive HTTPXRay console.

Syntax:
    python cli.py shell

Running cli.py with no command does the same thing:
    python cli.py


INTERACTIVE SHELL
=================

Launch:
    python cli.py

Prompt:
    httpxray>

Available shell commands:
    crawl
    stats
    list
    search
    export
    db-export
    db-import
    clear
    view
    help
    quit
    exit
    q


Shell command: help
-------------------
Syntax:
    help

Displays the interactive command summary.


Shell command: crawl
--------------------
Syntax:
    crawl <url> [workers] [depth] [external:y/n]

Defaults:
    workers = 10
    depth = 25
    external = no

Examples:
    crawl https://example.com

    crawl https://example.com 20

    crawl https://example.com 20 50

    crawl https://example.com 20 50 y

Accepted true values for the external argument:
    y
    yes
    1
    true


Shell command: stats
--------------------
Syntax:
    stats


Shell command: list
-------------------
Syntax:
    list [limit]

Examples:
    list
    list 250


Shell command: search
---------------------
Syntax:
    search <text>

Searches the HTTPXRay database and prints matching records.

Examples:
    search wordpress
    search application/json
    search .pdf


Shell command: export
---------------------
Syntax:
    export <csv|json|txt|html|pdf> <file> [filter]

Examples:
    export csv links.csv
    export html report.html
    export pdf report.pdf
    export html wordpress.html wordpress
    export json scripts.json javascript


Shell command: db-export
------------------------
Syntax:
    db-export <file.db>

Example:
    db-export saved-crawl.db


Shell command: db-import
------------------------
Syntax:
    db-import <file.db>

Example:
    db-import saved-crawl.db

The shell asks:
    Replace current HTTPXRay database? [y/N]:

Enter:
    y

to continue.


Shell command: clear
--------------------
Syntax:
    clear

The shell asks:
    Clear all stored links? [y/N]:

Enter:
    y

to continue.


Shell command: view
-------------------
Syntax:
    view [filter]

Creates links_view.html beside the application, then opens it in the browser.

Examples:
    view
    view wordpress
    view .css


Shell command: quit
-------------------
Any of these exits the interactive console:
    quit
    exit
    q


COMMON WORKFLOWS
================

Archive current crawl and start clean
-------------------------------------
python cli.py db-export site-before-clear.db
python cli.py crawl https://example.com --clear-first


Run a large internal crawl
--------------------------
python cli.py crawl https://example.com --workers 25 --depth 75


Include external links
----------------------
python cli.py crawl https://example.com --external --workers 20 --depth 40


Export a complete human-readable report
---------------------------------------
python cli.py export html httpxray-report.html


Export only matching records
----------------------------
python cli.py export html wordpress-links.html --filter wordpress


Use a separate database for a target
------------------------------------
python cli.py --db example-com.db crawl https://example.com --clear-first
python cli.py --db example-com.db stats
python cli.py --db example-com.db export html example-com.html


Restore an archived crawl
-------------------------
python cli.py db-import saved-crawl.db


View the database without crawling
----------------------------------
python cli.py stats
python cli.py list --limit 100
python cli.py view


HELP COMMANDS
=============

Main help:
    python cli.py --help

Subcommand help:
    python cli.py crawl --help
    python cli.py stats --help
    python cli.py list --help
    python cli.py export --help
    python cli.py db-export --help
    python cli.py db-import --help
    python cli.py clear --help
    python cli.py view --help
    python cli.py shell --help


DEPENDENCIES
============

Core:
    httpx
    beautifulsoup4

PDF export:
    reportlab

SQLite is provided by Python's standard library.


FILES
=====

cli.py
    HTTPXRay 2.0 command-line application.

httpxray.db
    Default live SQLite crawl database.

links_view.html
    Default browser-view output created by the view command.

User-defined exports and database archives may be written anywhere permitted by the operating system.
