Introduction: What Is SiteWatcher?
With the rapid pace of digital information, staying informed of important changes on websites has become a critical need for businesses and individuals alike. Whether it's ensuring you're the first to know about website updates, sales announcements, or crucial maintenance alerts, having a tool to monitor websites is essential. Enter SiteWatcher, an innovative website monitoring service that keeps users updated whenever significant changes occur on specified websites.
SiteWatcher is designed to crawl websites periodically and alert users through emails when particular conditions are met, such as new content, specific keyword mentions, or the absence of expected phrases. The service is versatile, offering everything from simple change detection to advanced regular expression (regex) matching. Its powerful features and ease of use make it one of the most practical tools for keeping track of important updates on websites.
This detailed guide will take you through how SiteWatcher works, its features, and technologies, and why it is a must-have tool for anyone who wants to keep an eye on online changes.
Why SiteWatcher Is Needed: The Motivation Behind the Tool
The internet has grown into an indispensable part of our daily lives, and with it, the importance of being notified when critical changes happen has risen. Imagine waiting for a website to announce product availability or updates to a service you rely on, only to miss the notice because you weren't consistently checking the site. Or worse, losing water service because your water provider didn’t send an alert about a maintenance event. This is precisely where SiteWatcher proves its value.
The original motivation behind SiteWatcher was driven by real-world challenges. In some regions, services such as water supply or electricity might be disrupted due to maintenance without prior email alerts from the service providers. SiteWatcher aims to fill this communication gap by monitoring websites for updates and notifying users the moment a keyword or change is detected.
For businesses, it can be crucial to monitor competitors' websites for new offerings, content changes, or policy updates. Similarly, individuals can use SiteWatcher to track website content, ensuring they don’t miss out on events like sales, job postings, or important announcements.
Key Features of SiteWatcher
1. Customizable Website Monitoring
SiteWatcher allows users to choose how they want to monitor a website. Whether it's for any change on the site, specific keywords appearing, or a regex match on the HTML structure of the page, SiteWatcher has you covered. Users can specify what kind of notification they want to receive, whether it's for small changes or substantial updates.
2. Frequency of Monitoring
The application offers flexibility in how often it checks for updates. Users can set a monitoring frequency ranging from every 2 hours to every 24 hours. This means SiteWatcher will automatically crawl the website at the specified interval, ensuring that you are promptly informed when changes happen.
3. Google Login and Security Features
SiteWatcher integrates Google Login using OAuth Code Flow with PKCE (Proof Key for Code Exchange). This ensures secure authentication for users, eliminating the need to create separate accounts and managing passwords. PKCE adds layer of security, reducing the risk of compromised credentials.
4. Grouped Notifications
To prevent email spam, SiteWatcher intelligently groups alerts. Instead of bombarding you with individual notifications for every small change detected, the service consolidates updates and sends a single email summary based on your monitoring frequency. This means fewer notifications cluttering your inbox, while still ensuring you are up-to-date.
5. Device Control
You have complete control over your devices when using SiteWatcher. With a simple click, you can disconnect all your devices from the application, ensuring your account remains secure and that no unwanted device access goes unnoticed.
Technologies Powering SiteWatcher
SiteWatcher is built on cutting-edge technologies that ensure reliability, speed, and scalability. The current stack includes:
Frontend: Angular 13
Backend: .NET 7 with Entity Framework Core 7
Database: PostgreSQL
Message Queuing: Masstransit
Caching and Rate Limiting: Redis
Testing Frameworks: xUnit, Moq, FluentAssertions
Web Scraping: AngleSharp
Validation: FluentValidation
Mailing Service: MailKit
Other Technologies: Scrutor, Dapper, Polly, BenchmarkDotNet, Test containers, Roslynator
The combination of these technologies allows SiteWatcher to be fast, reliable, and efficient in monitoring websites and sending notifications. The usage of modern frameworks ensures that the application can scale with growing user demand while maintaining a strong focus on security and performance.
Backend Architecture of SiteWatcher
SiteWatcher employs a pragmatic backend architecture inspired by concepts of Domain-Driven Design (DDD). The architecture is layered, adhering to the principles of hexagonal architecture (also known as ports and adapters architecture). This modular structure allows the business logic to remain isolated from the technical infrastructure, enabling flexibility in adding or modifying functionalities.
SiteWatcher's current architecture includes:
Aggregates: These are the larger building blocks responsible for handling business logic. For example, the Notification aggregate handles creating notifications, and its associated entities handle the smaller details such as specific alerts.
Message Queue: SiteWatcher uses MediatR to send domain events, which are then processed asynchronously by a background worker. These events include actions like sending email notifications, user confirmations, and periodic checks on websites.
Retry Policy: The system uses Polly for retry logic, ensuring transient errors (like network timeouts) don’t prevent critical tasks from completing. This is especially useful when crawling websites that may temporarily be unavailable.
Notification System: Alerts are grouped, processed, and queued for email delivery using a worker service. This outbox pattern guarantees that notifications are sent without duplication, providing a seamless user experience.
How SiteWatcher Works: A Step-by-Step Overview
User Setup: Users log in with Google and set up their website monitoring preferences.
Site Crawling: At the specified monitoring interval (e.g., every 4 hours), SiteWatcher's crawler checks the selected websites for any changes, specific keywords, or regex matches.
Event Triggering: When a monitored condition is met (e.g., a keyword is detected), the system triggers an AlertTriggeredEvent, which gets published to a queue.
Notification Creation: The background worker consumes the event, creates a notification, and queues it for email delivery.
Email Dispatch: The queued notifications are processed, and a grouped alert email is sent to the user. This email contains all the updates for the given monitoring period.
User Control: Users can monitor their notifications, adjust monitoring rates, or disconnect devices as needed.
Why SiteWatcher Stands Out from the Competition
There are plenty of website monitoring tools available, but SiteWatcher offers a unique set of advantages that set it apart:
Precision Alerts: Users can configure highly specific notifications based on keywords, regex patterns, or simple change detection.
Minimal Email Spam: Unlike many other monitoring services that send numerous emails, SiteWatcher uses a consolidated email system that minimizes clutter while ensuring all essential alerts are delivered.
Strong Security Measures: With Google OAuth integration and PKCE, SiteWatcher offers a high level of account security.
Customizable Monitoring Rates: Users can decide how often websites are checked, offering more control than most similar services.
Common Use Cases for SiteWatcher
1. Monitor Competitor Websites
Businesses can use SiteWatcher to track changes on competitors’ websites, ensuring they’re always aware of new products, services, or announcements.
2. Get Notified of Service Disruptions
Users can monitor local service providers’ websites (e.g., water or electricity companies) to receive alerts when maintenance is scheduled or outages are expected.
3. Track Sales and Discounts
For online shoppers, SiteWatcher can be configured to send alerts when favorite products go on sale or new deals are posted on eCommerce sites.
4. Job Listings
Job seekers can use SiteWatcher to monitor career pages or job boards for specific keywords like "job openings" or "new postings."
FAQs About SiteWatcher
Q1: What is SiteWatcher used for?
A: SiteWatcher is a website monitoring tool that alerts users when specific changes occur on a webpage, such as content updates, keyword mentions, or regex matches.
Q2: How does SiteWatcher notify me of changes?
A: SiteWatcher sends notifications via email, summarizing all changes detected within a selected time frame based on your monitoring frequency settings.
Q3: Can I use SiteWatcher without creating a new account?
A: Yes, SiteWatcher uses Google OAuth for secure login, meaning you can log in with your Google account without needing to create a new password.
Q4: Is SiteWatcher free to use?
A: SiteWatcher’s pricing model has not been fully disclosed yet, as it is still under development. However, it will likely offer a freemium model with basic and advanced features.
Q5: Does SiteWatcher support monitoring multiple websites?
A: Yes, SiteWatcher supports monitoring multiple websites simultaneously, allowing users to track various pages for different changes.
Q6: What makes SiteWatcher secure?
A: SiteWatcher uses OAuth Code Flow with PKCE, ensuring secure login processes and reducing the risk of unauthorized access. Additionally, the service supports device management features for user control.
Q7: How frequently can I monitor a website with SiteWatcher?
A: SiteWatcher offers flexible monitoring rates, allowing users to check websites at intervals ranging from every 2 hours to every 24 hours.
Q8: Does SiteWatcher prevent duplicate emails?
A: Yes, SiteWatcher uses an outbox pattern to ensure email alerts are not duplicated, even when multiple changes are detected within the same monitoring period.
Conclusion: SiteWatcher – Your Go-To Tool for Website Monitoring
In a world where staying updated on website changes can be crucial, SiteWatcher offers a powerful and customizable solution. Whether you need to monitor service disruptions, track competitor activities, or get notified of sales, SiteWatcher has you covered. Its flexibility, strong security features, and advanced notification system make it a standout tool for individuals and businesses alike.
With SiteWatcher, you’ll never miss critical updates from the websites that matter most to you. This tool simplifies the process of website monitoring while ensuring that your inbox remains clutter-free. Start using SiteWatcher today and experience the benefits of real-time website alerts without the hassle of manual monitoring.
Key Takeaways:
SiteWatcher is a website monitoring tool that sends email alerts for website changes, keyword mentions, or regex matches.
It offers customizable monitoring frequencies ranging from 2 to 24 hours.
Google OAuth integration ensures secure login without needing a separate password.
The tool reduces email clutter by grouping notifications into a single summary email.
Security is enhanced with PKCE, outbox patterns, and device management features.
Ideal for businesses tracking competitors, individuals monitoring services, or shoppers looking for sales alerts.
留言