# FALCON - Fetch and Log Continuous Observations of Networks FALCON is a tool for continuously fetching websites and logging performance metrics like page load time, element rendering, and more. It provides a flexible way to monitor websites and PNG file performance over time, with customizable sleep intervals and automatic logging of metrics. ## Features - **Continuous Website and PNG Fetching**: Continuously fetches a list of websites or PNG files, logs the time it takes to load the entire page, and allows for the waiting of specific elements to fully render. - **Customizable Sleep Intervals**: Allows users to define a minimum and maximum sleep time between requests to prevent constant polling. - **Save and Load Configuration**: Automatically saves the user’s input (URLs, specific elements to wait for, sleep intervals) into a configuration file (`config.json`) and loads it on startup. - **Metrics Collection**: Logs the time for a full page load and specific elements to become visible, and provides detailed metrics like minimum, maximum, and average load times for each URL. - **CSV Logging**: Saves the performance data in a CSV file (`fetch_times.csv`), making it easy to analyze trends and performance over time. ## How It Works ### Fetch and Load Mechanism FALCON uses **Chromedp** to load websites and PNG files, and it operates by: 1. **Fetching PNG Files**: - For PNG files, the fetch mechanism simply requests the file and logs the time it took to fully receive it. 2. **Fetching Websites**: - For websites, FALCON waits for the entire page to load, including all static assets (such as images, CSS, JavaScript) by waiting for the `load` event. - FALCON also waits for a **user-specified element** to become visible. This ensures that dynamic content (like JavaScript-rendered elements) is fully loaded before calculating the total time. - FALCON adds a small buffer by waiting for **network idle** conditions to ensure all asynchronous loading is completed. ### Collected Metrics 1. **Full Page Load Time**: - The time from the start of the navigation to the point where the entire page (including all images, stylesheets, and scripts) is loaded. 2. **Element Render Time**: - The time it takes for a specific element (e.g., footer, main content) to become visible. This can be useful for monitoring dynamically loaded content. 3. **Sleep Between Fetches**: - A user-specified **min** and **max** sleep interval is used to control how often FALCON fetches the URLs, ensuring it doesn’t overload the server with constant requests. 4. **Minimum, Maximum, and Average Load Time**: - FALCON calculates the minimum, maximum, and average load time for each URL, providing insight into the performance over multiple fetch attempts. ## Installation ### Prerequisites - **Go** (version 1.19 or higher) - **Chromedp**: FALCON uses Chromedp, a Go library for driving headless Chrome. You can install Chromedp by running: ```bash go get -u github.com/chromedp/chromedp ``` ### Build and Run To build and run FALCON from source: ```bash git clone https://gitlab.com/<your-username>/<your-repo>.git cd <your-repo> go build -o falcon ./falcon ``` For specific platforms: - **Windows**: `GOOS=windows GOARCH=amd64 go build -o falcon.exe` - **Linux**: `GOOS=linux GOARCH=amd64 go build -o falcon` - **Mac**: `GOOS=darwin GOARCH=amd64 go build -o falcon` ## Usage 1. Enter URLs: Provide a list of URLs (one per line) in the URL input field. 2. Enter CSS Selector: Specify a CSS selector (e.g., #footer, .content) that should be visible before calculating the load time. 3. Set Sleep Intervals: Define the minimum and maximum sleep intervals (in seconds) between fetch attempts. 4. Start: Click the Start button to begin fetching the URLs continuously. 5. Stop: Click the Stop button to halt the fetching process. ## Configuration File FALCON automatically saves your input into a configuration file (config.json) when you start fetching. On subsequent runs, this configuration will be loaded automatically, pre-filling the input fields. The config.json file has the following structure: ```json { "urls": ["https://example.com", "https://another.com"], "element": "#footer", "min_interval": 5, "max_interval": 10 } ``` You can manually edit this file to change the URLs, element, or intervals. ## Metrics Example The following metrics are collected for each URL: - Min Load Time: The shortest time taken to load the full page. - Max Load Time: The longest time taken to load the full page. - Average Load Time: The average time across all fetches. The CSV file (fetch_times.csv) will contain logs like this: ```csv Timestamp,URL,Load,TTFB,LCP 2024-10-05T12:34:56Z,https://example.com,2.34s,1.23s,1.45s 2024-10-05T12:36:00Z,https://example.com,2.11s,1.12s,1.34s 2024-10-05T12:37:30Z,https://another.com,1.45s,0.89s,1.12s ``` ## Download You can download the latest release for your platform below: - [Windows version](https://gitlab.ard.de/apps/falcon/-/releases/permalink/latest/download/falcon_windows.exe) - [macOS version](https://gitlab.ard.de/apps/falcon/-/releases/permalink/latest/download/falcon_mac) - [Linux version](https://gitlab.ard.de/apps/falcon/-/releases/permalink/latest/download/falcon_linux) ## License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.