Daily Fuel Price Tracker

Project Purpose & Overview

Fuel prices across municipal and regional distribution networks experience frequent daily volatility. For fleet managers, logistics operators, and purchasing analysts, tracking these price shifts across dozens of cities manually is unfeasible.

This system was built to provide automated daily pricing intelligence by autonomously querying official tax invoice streams (SEFAZ-MT Menor Preço), parsing historical price quotes, filtering non-station noise and statistical outliers, and storing enriched time-series data for analytics and decision-making.

Architecture & Pipeline

• Multi-City Scheduled Extraction: Automated crawlers querying price data across 17+ major municipalities (Cuiabá, Várzea Grande, Rondonópolis, Sinop, Sorriso, Cáceres, Tangará da Serra, etc.).
• Multi-Fuel Classification: Categorizes and tracks Ethanol (Etanol Comum/Aditivado), Gasoline (Gasolina Comum/Aditivada), and Diesel (S10/S500).
• Smart Entity Validation & Outlier Rejection: Regex pattern matching filters out non-fuel businesses (auto parts, repair shops, bakeries) and enforces statistical sanity checks to eliminate pricing anomalies.
• Composite Key Deduplication: Generates unique identifiers based on timestamp + CNPJ + product code to ensure idempotent ingestion.
• Cloud Database Persistence: Real-time streaming into Firebase Firestore and structured relational MySQL schema for high-throughput time-series analysis.
• Automated CI/CD Workflows: Runs hands-free via GitHub Actions workflows and includes continuous local loop scheduling modes.

Technical Highlights