Black-Friday Black-Friday
View all actors
logo

Content Checker - Scrape & Monitor Website Content

RealdataAPI / content-checker

Scrape and monitor website content for changes on web pages. Automatically store after and before snapshots and get email alerts using the content checker. Use the content-checking tool in countries like USA, UK, UAE, France, Australia, Germany, Spain, Singapore, Mexico, and more.

Customize me! Report an issue E-commerce

What is the Working of the Content Checker?

The content scraper allows you to track website content for any web page. After checking the content for changes, the scraper sends an email alert with after and before screenshots. Use these screenshots and alerts to curate your watchdog for product sales, updates, prices, and competitors or monitor website content changes from selected web pages.

Technically, it scrapes the website content of the textual form using the selector and compares it with the past execution. It runs another scraper to store screenshots and sends them in the email.

Input Of the Content Checker

The content scraping tool needs a content selector, URL, and email id as the input to scrape the website content. Further, you can define a screenshot selector. Alternatively, you can choose a content selector to choose screenshots.

Check out the input tab to learn more about the detailed input description.

The Output of the Content Checker

After execution, the scraper will update the content and screenshots in the key-value store related to the scraper task.

If there is a change in content, the content checker will call another scraper to send email alerts.

Check out the below example of an email alert with changed and previous content with screenshots:

Content Checker with Integrations

You can connect the content checker with any web application or cloud service with the help of integrations available on our platform. Further, you can integrate it with Zapier, Make, Airbyte, Google Drive, Google Sheets, Slack, GitHub, and more. The content-checking tool also allows you to use webhooks to take action for the event's commencement. For example, you can receive an alert after the successful execution of the content-scraping tool.

Using Content Checker with Real Data API

Our scraper gives programmatic permission to access the platform. We have organized it around RESTful HTTP endpoints to allow you to schedule, manage and run the scrapers available on our platform. Real Data API also allows you to retrieve results, scraper performances, update and create scraper versions, access datasets, etc. Use our client NPM package and Client PyPl package to access the scraper using Node.js and Python, respectively.

Need Help Getting Expected Outputs? Develop Customized Scraper

If the content checker can't deliver what you want, you can develop a customized scraper according to your requirements. Multiple scraper templates on our platform support TypeScript, Python, and JavaScript, to begin with. Besides, according to your requirements, you can directly write the code using Crawlee, the open-source library.

If you want to avoid developing it by yourself, contact us for a customized solution for scraping.

Your Feedback on Content Checker

Our team is constantly working on the performance improvement of the scraper. Therefore, if you want to suggest anything or report any bug, please create an issue from the issue tab, or mail us about it.

Industries

Check out how industries are using Content Checker around the world.

saas-btn.webp

E-commerce & Retail

To run the code examples, you need to have an RealdataAPI account. Replace < YOUR_API_TOKEN> in the code with your API token.

import { RealdataAPIClient } from 'RealdataAPI-Client';

// Initialize the RealdataAPIClient with API token
const client = new RealdataAPIClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare actor input
const input = {
    "url": "https://www.RealdataAPI.com/change-log",
    "contentSelector": "[class^=change-log__MonthBox-]:nth-of-type(1) ul",
    "screenshotSelector": "[class^=change-log__MonthBox-]:nth-of-type(1) ul",
    "sendNotificationText": "RealdataAPI found a new change!",
    "proxy": {
        "useRealdataAPIProxy": false
    },
    "navigationTimeout": 30000
};

(async () => {
    // Run the actor and wait for it to finish
    const run = await client.actor("jakubbalada/content-checker").call(input);

    // Fetch and print actor results from the run's dataset (if any)
    console.log('Results from dataset');
    const { items } = await client.dataset(run.defaultDatasetId).listItems();
    items.forEach((item) => {
        console.dir(item);
    });
})();

from RealdataAPI_client import RealdataAPIClient

# Initialize the RealdataAPIClient with your API token
client = RealdataAPIClient("<YOUR_API_TOKEN>")

# Prepare the actor input
run_input = {
    "url": "https://www.RealdataAPI.com/change-log",
    "contentSelector": "[class^=change-log__MonthBox-]:nth-of-type(1) ul",
    "screenshotSelector": "[class^=change-log__MonthBox-]:nth-of-type(1) ul",
    "sendNotificationText": "RealdataAPI found a new change!",
    "proxy": { "useRealdataAPIProxy": False },
    "navigationTimeout": 30000,
}

# Run the actor and wait for it to finish
run = client.actor("jakubbalada/content-checker").call(run_input=run_input)

# Fetch and print actor results from the run's dataset (if there are any)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# Set API token
API_TOKEN=<YOUR_API_TOKEN>

# Prepare actor input
cat > input.json <<'EOF'
{
  "url": "https://www.RealdataAPI.com/change-log",
  "contentSelector": "[class^=change-log__MonthBox-]:nth-of-type(1) ul",
  "screenshotSelector": "[class^=change-log__MonthBox-]:nth-of-type(1) ul",
  "sendNotificationText": "RealdataAPI found a new change!",
  "proxy": {
    "useRealdataAPIProxy": false
  },
  "navigationTimeout": 30000
}
EOF

# Run the actor
curl "https://api.RealdataAPI.com/v2/acts/jakubbalada~content-checker/runs?token=$API_TOKEN" /
  -X POST /
  -d @input.json /
  -H 'Content-Type: application/json'

URL to Check

url Required String

Choose the webpage URL you want to monitor.

Monitored Area Selector

contentSelector Required String

It is the CSS selector of the target area you want to track.

Screenshot Selector

screenshotSelector Optional String

It is the CSS selector to take screenshots.

Email Address

sendNotificationTo Required String

Enter your email ID to receive the notification.

Notification Text

sendNotificationText Optional String

It is an optional text to add to the email alert.

Error Notification

informOnError Optional Enum

If there is any error with any selector on the webpage, you will receive an email alert with screenshots.

Options:

false string, true string

Proxy Server Configuration

proxy Optional Object

Choose the relevant proxy server if the source website blocks the scraper or IP address.

Navigation Timeout

navigationTimeout Optional Integer

It is the millisecond duration the page should wait to time out.

How to Retry

retryStrategy Optional Enum

Sometimes, the webpage fails to load correctly, or the source website blocks the scraper. Besides, more than retrying the inaccurate selector may need. However, the blocked page recognition is not 100 percent accurate.

Options:

never-retry string, on-all-errors string, on-block string

Maximum Count of Retries

maxRetries Optional Integer

It is to check how often the scraper must retry the process if there are errors. 

{
  "url": "https://www.RealdataAPI.com/change-log",
  "contentSelector": "[class^=change-log__MonthBox-]:nth-of-type(1) ul",
  "screenshotSelector": "[class^=change-log__MonthBox-]:nth-of-type(1) ul",
  "sendNotificationText": "RealdataAPI found a new change!",
  "informOnError": "false",
  "proxy": {
    "useRealdataAPIProxy": false
  },
  "navigationTimeout": 30000,
  "retryStrategy": "on-block",
  "maxRetries": 5
}
Real Data API stands out as one of the premier global companies specializing in web data scraping and web extraction.

INDUSTRIES

E-commerce Data Scraping

Food Data Scraping

Grocery Data Scraping

Travel & Logistics

Liquor Data Scraping

Social Media Data Scraping

Software & SaaS Data Scraping

Telecommunications Data Scraping

 

Real estate Data Scraping

Research & Education Data Scraping

Fintech & Insurance Data Scraping

NGO & Government Data Scraping

Consulting Data Scraping

Manufacturing Data Scraping

Healthcare Data Scraping

API Store

Amazon Product Scraper

AliExpress Scraper

Google Shopping Scraper

Zomato Scraper

Swiggy Scraper

E-Commerce Scraper

Amazon Scraper

Flipkart Scraper

Twitter Scraper

eBay Scraper

WooCommerce Scraper

Google Maps Scraper

Airbnb scraper

Booking scraper

Tiktok Scraper

Reddit Scraper

Use Cases

Price Comparison

Market Research

Robotic Process Automation (RPA)

Product Development

Lead Generation

Machine Learning

Web Data

Live Crawler

Use Cases

Web Scraping

Enterprise Web Crawling

Web Scraping API Services

Mobile App Scraping services
clutch
clutch
clutch
clutch
clutch
company widget

About Us

Contact us
© 2024 RealdataAPI. All rights reserved.

Disclaimer : Real Data API only extracts publicly available data while maintaining a strict policy against collecting any personal or identity-related information.