What does Reddit Web Data Scraper do?
An unofficial Reddit API collects unlimited data from Reddit without authentication. It allows you to extract comments and posts together along with some information about the user without login. We have developed it with Real Data API SDK; you can use it locally or on our platform.
Reddit Scraper permits you to:
- Scrape top posts from subreddits with community details, including member count, moderator username, category, and URL.
- Scrape the famous subreddits and leaderboards.
- Get the Reddit timestamp, usernames, comments, points, posts, and comments URL.
- Sort extracted data by categories of relevance like Top, Hot, New, etc.
- Scrape the latest posts, comments, and respective user details.
- Scrape Reddit data using specific keywords or URLs.
Do You Need Only a Few Results From Reddit?
Try our dedicated free Reddit Scraper if you want to extract Reddit data quickly on a smaller scale. Only enter keywords or Reddit URLs and tap on the scrape option. Remember that free Reddit Scraper can scrape up to 10 comments, 10 posts, 2 leaderboard items, and 2 subreddits for you.
What is the Cost of Using Reddit Scraper?
Reddit Scraper on the Real Data API platform gives you one thousand results for 4 USD platform credits. You can cover it with our free 5 USD platform credit in our monthly free plan.
However, if you want to scrape more Reddit data, you must try our monthly personal plan of 49 USD to get over ten thousand results in a month.
How to Scrape Data From Reddit?
You don't need any coding knowledge or skill to use Reddit Data Scraper API. If you don't know where to begin, follow the below stepwise video tutorial. You can also use this tutorial for Free Reddit Scraper.
How to Use Scraped Data From Reddit?
- Research your favorite topics and opinions from a wide range of audiences.
- Monitor discussions about your products and brand across relevant subreddits.
- Track debates over subjects with high stakes like new technologies, political general news, finance, and more.
- Track your business mentions or your favorite topic automatically.
- Explore the latest trends, PR opportunities, and attitudes.
- Search and scrape Reddit comments to begin and support the sentiment analysis.
Input Parameters
There are two methods to scrape Reddit if you run Reddit Scraper on the Real Data API platform.
- Using the Start URLs input field -It will collect all the details from any Reddit URL to collect user, post, or community data.
- Or using the Search Term input field - It will crawl all the Reddit data, including posts, communities, and users for particular search keywords.
How to scrape Reddit data by URLs
Almost any link from Reddit will return a dataset. The scraper will display a message if the URL is not supported before scraping the page.
Input Examples:
These are a few input examples of Reddit URLs that you can scrape.
Note: The scraper will only scrape posts if you use the search link as a startURL parameter. Use the specific URL or search field for Reddit user search and community search.
Scraping Reddit using Search Term
Search Types: It denotes which part of Reddit you are scraping - users, communities, or posts.
Search Term: It is the keyword you want to search on the Reddit search engine. You can add multiple fields or keep only one. If you are using startUrls, don't try this.
Sort Search: It will sort Search outputs by Top, Hot, most comment counts, or Relevance.
Filter by Time or Date: It will categorize the search by the last month, day, week, hour, or year. You can use it only while scraping posts.
To check the entire parameter list, how to set default values, and actual default values, go to the Input Schema tab.
Input Example:
It is an input example of how the scraper will display the input field if you scrape all Reddit users and communities with the keyword parrot. You will see the sorted output by the latest first.
{
"maxItems": 10,
"maxPostCount": 10,
"maxComments": 10,
"maxCommunitiesAndUsers": 10,
"maxLeaderBoardItems": 10,
"scrollTimeout": 40,
"proxy": {
"useRealdataAPIProxy": true
},
"debugMode": false,
"searches": ["parrots"],
"type": "communities_and_users",
"sort": "new",
"time": "all"
}
Results
The scraper will store the output in the dataset. The dataset contains each comment, community, list, or user. Once the Reddit API finishes the run, you can export the scraper Reddit data on your device or export it to any web application in multiple usable formats. Check out the below output examples for various input examples.
Example Reddit Post
{
"id": "ss5c25",
"title": "Weekly Questions Thread / Open Discussion",
"description": "For any questions regarding dough, sauce, baking methods, tools, and more, comment below.You can also post any art, tattoos, comics, etc here. Keep it SFW, though.As always, our wiki has a few sauce recipes and recipes for dough.Feel free to check out threads from weeks ago.This post comes out every Monday and is sorted by 'new'.",
"numberOfVotes": "4",
"createdAt": "3 days ago",
"scrapedAt": "2022-01-09T22:52:48.489Z",
"username": "u/AutoModerator",
"numberOfComments": "19",
"mediaElements": [],
"tag": "HELP",
"dataType": "post"
}
Example Reddit Comment
{
"url": "https://www.reddit.com/r/Pizza/comments/sud2hm/tomato_pie_from_sallys_apizza_stamford_ct/t1_hx9k9it",
"username": "Acct-404",
"createdAt": "9 h ago",
"scrapedAt": "2022-03-09T12:52:48.547Z",
"description": "Raises handUhhhh can I get some cheese on my pizza please?",
"numberOfVotes": "3",
"postUrl": "https://www.reddit.com/r/Pizza/comments/sud2hm/tomato_pie_from_sallys_apizza_stamford_ct/",
"postId": "sud2hm",
"dataType": "comment"
}
Example Reddit Community
{
"title": "Pizza",
"alternativeTitle": "r/Pizza",
"createdAt": "Created Aug 26, 2008",
"scrapedAt": "2022-03-09T12:54:42.721Z",
"members": 366000,
"moderatos": [
"6745408",
"AutoModerator",
"BotTerminator",
"DuplicateDestroyer"
],
"url": "https://www.reddit.com/r/pizza/",
"dataType": "community",
"categories": ["hot", "new", "top", "rising"]
}
Notes for Developers
Limiting outputs with maxItems
You can set up the maximum ist count you want to scrape the user or inside the community if you need to restrict the search scope. Further, using the parameters below, you can restrict the comment count for every post and community count with the leaderboard numbers.
{
"maxPostCount": 50,
"maxComments": 10,
"maxCommunitiesAndUsers": 5,
"maxLeaderBoardsItems": 5
}
If you want to prevent a long actor run, you can set max items. Once it reaches the result count you have asked to scrape, it will stop the scraper. Hence you should take care not to trim your outputs.
Visit the Input schema tab to check the entire list of methods to limit Reddit Scraper with maxLeaderBoardItems, maxComments, maxItems, maxCommunitiesAndUsers, and maxPostCount.
Extend Output Function
You can use this scraper function to update the output results of this scraper. You can select the data type you want to scrape from Reddit. The resulting output will merge with the output from this function.
To achieve three different things, you can return the below fields.
- Remove a field: return the existing field with an undefined value.
- Add a new field - return the field with an object without the resulting output.
- Change a field - use a new value to return the existing field.
async () => {
return {
pageTitle: document.querySelector("title").innerText,
};
};
The below example will add the page title to the final object.
{
"title": "Pizza",
"alternativeTitle": "r/Pizza",
"createdAt": "Created Aug 26, 2008",
"scrapedAt": "2022-03-08T21:57:25.832Z",
"members": 366000,
"moderators": [
"6745408",
"AutoModerator",
"BotTerminator",
"DuplicateDestroyer"
],
"url": "https://www.reddit.com/r/pizza/",
"categories": ["hot", "new", "top", "rising"],
"dataType": "community",
"pageTitle": "homemade chicken cheese masala pasta"
}
