Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is RealdataAPI's basic tool for web crawling and scraping.
It is a data scraping tool to scrape gaming product data with accessory data from PlayStation Store. Since the PlayStation platform has no official data API, our gaming accessory data scraper should help you collect the required data.
Our PlayStation Store data scraper has the following features:
Our team is actively developing this scraper. If you want to suggest any feature or fix any bug, please create an issue from the issue tab of your console account.
It would be helpful if you needed the JSON formatted input to the PlayStation Store Data Scraper. Here are the possible input parameters:
You have to use proxies from multiple options, like your proxy or that from our server.
To scrape the PlayStation gaming accessory and hardware data from a particular listing URL, copy the link and paste it as the starting URL.
Enter the URL for the first page and keep endPage as one to scrape only the first page from the list.
Further, you can retrieve any page intervals using the above approach. For example, if you provide the 7th page and keep endPage as 8, you will get only the 7th and 8th-page data.
We have optimized the scraper to run fast and extract the maximum possible data. Hence, the scraper forefronts every request for listing details. If the source blocks the scraper infrequently, it will give you a hundred listings in 120 seconds. It will cost you around 0.07 to 0.09 compute units.
{ "startUrls":[ "https://store.playstation.com/en-tr/product/EP4389-CUSA17440_00-DCLGAMEEU0000000", "https://store.playstation.com/en-tr/search/game", "https://store.playstation.com/en-tr/concept/10000886", "https://store.playstation.com/en-tr/category/d42c9dc6-5516-4a34-a511-c09894266d98/1" ], "search":"food game", "includeReviews": false, "proxy": { "useRealdataAPIProxy": true }, "endPage": 5, "maxItems": 100 }
While executing, the scraper will send alerts to inform what it is scraping. The scraper will mention which page it is currently scraping from the list. After loading items from the list, you will see the event message with the total and loaded item counts for every page.
The scraper will only work if you give the correct input and display the failure reason with an explanation.
The scraper saves its output in a dataset. Every item has a unique location and value in the dataset.
You can get the output in any programming, like Node.js, Python, NPM, PHP, etc.
You will see the scraped PlayStation Store data in the following format:
{ "url": "https://store.playstation.com/en-tr/product/EP9000-BCES00850_00-LBPDLCORIGCO0083", "id": "EP9000-BCES00850_00-LBPDLCORIGCO0083", "privacyPolicy": null, "publisherName": "Sony Interactive Entertainment Europe", "descriptions": [ { "type": "LONG", "subType": "NONE", "value": "Game For Anything<br>Turn your Sackboy® into a creative genius with this Game Writer costume, complete with a fashion-statement moustache and ironic T-shirt.<br><br>Sackboy® Says:<br>• This costume is also available to download from PlayStation®Store in the Sackboy's Casual Friday Costume Pack.<br><br>This add-on is for LittleBigPlanet™ 2.<br><br>Buy this add-on for LBP™ 2 and get the LittleBigPlanet™ Karting, LittleBigPlanet™ PlayStation®Vita and LittleBigPlanet™ 3 (PS4™ and PS3™) versions at NO EXTRA COST.<br><br>After purchase, open the PlayStation®Store “Download List” to find this add-on ready to be downloaded.<br><br>1-4 Players, 1300MB minimum space required, HDTV screen resolution: 720p, Network Features, Network Players 2-4, PlayStation®Move Optional<br><br>Download of this product is subject to the Sony Entertainment Network Terms of Service/User Agreement and any specific additional conditions applying to this product. If you do not wish to accept these terms, do not download this product. See Terms of Service for more important information.<br> PS4: One-time licence fee to download to multiple PS4 systems. Sign in to PSN is not required to use this on your primary PS4, but is required for use on other PS4 systems.<br>PS3: One-time fee for use of downloads on up to 2 activated PS3 systems.<br>PS Vita: One-time fee for use of downloads on up to 3 activated compatible Portable Console systems.<br>See Health Warnings for important health information before using this product.<br>Library programs ©Sony Computer Entertainment Inc. exclusively licensed to Sony Computer Entertainment Europe. Software Usage Terms apply, See eu.playstation.com/legal for full usage rights.<br><br>LittleBigPlanet™ 2 ©2010 Sony Computer Entertainment Europe. Published by Sony Computer Entertainment Europe. Developed by Media Molecule. “LittleBigPlanet”, “LittleBigPlanet logo”, “Sackboy” and “Sackgirl” are trademarks or registered trademarks of Sony Computer Entertainment Europe. All rights reserved." } ], "localizedGenres": null, "releaseDate": "2013-07-31T00:00:00Z", "spokenLanguages": [], "screenLanguages": [], "platforms": [ "PS4" ], "contentRating": { "authority": "PEGI", "description": "PEGI 7+", "name": "PEGI_7", "url": "https://image.api.playstation.com/grc/images/ratings/hd/pegi/7.png", "interactiveElements": [], "descriptors": [ { "description": "Fear", "name": "PEGI_FEAR", "url": "https://image.api.playstation.com/grc/images/descriptors/hd/pegi/fear.png" }, { "description": "Online", "name": "PEGI_ONLINE", "url": "https://image.api.playstation.com/grc/images/descriptors/hd/pegi/online.png" }, { "description": "Violence", "name": "PEGI_VIOLENCE", "url": "https://image.api.playstation.com/grc/images/descriptors/hd/pegi/violence.png" } ] }, "name": "Game Writer Costume", "concept": null, "skus": [ { "__ref": "Sku:EP9000-BCES00850_00-LBPDLCORIGCO0083-E001" }, { "__ref": "Sku:EP9000-BCES00850_00-LBPDLCORIGCO0083-E002" } ], "isAgeRestricted": false, "activeCtaId": "ADD_TO_CART:ADD_TO_CART:EP9000-BCES00850_00-LBPDLCORIGCO0083-E001:OUTRIGHT", "webctas": [ { "__ref": "GameCTA:ADD_TO_CART:ADD_TO_CART:EP9000-BCES00850_00-LBPDLCORIGCO0083-E001:OUTRIGHT" } ], "isInWishlist": false, "isWishlistable": true, "edition": null, "media": [ { "type": "IMAGE", "role": "MASTER", "url": "https://image.api.playstation.com/cdn/EP9000/BCES00850_00/cbixxOGGNeGKWQpapaWWQ2HKRaSQP3NA.png" } ], "prices": [ "4,50 TL" ], "editions": [] }
You should have a Real Data API account to execute the program examples. Replace
< YOUR_API_TOKEN>
in the program using the token of your actor. Read about the live APIs with Real Data API docs for more explanation.
import { RealdataAPIClient } from 'RealdataAPI-Client';
// Initialize the RealdataAPIClient with API token
const client = new RealdataAPIClient({
token: '<YOUR_API_TOKEN>',
});
// Prepare actor input
const input = {
"search": "food game",
"startUrls": [
"https://store.playstation.com/en-tr/product/EP4389-CUSA17440_00-DCLGAMEEU0000000",
"https://store.playstation.com/en-tr/search/game",
"https://store.playstation.com/en-tr/concept/10000886",
"https://store.playstation.com/en-tr/category/d42c9dc6-5516-4a34-a511-c09894266d98/1"
],
"maxItems": 20,
"endPage": 1,
"extendOutputFunction": ($) => { return {} },
"customMapFunction": (object) => { return {...object} },
"proxy": {
"useRealdataAPIProxy": true
}
};
(async () => {
// Run the actor and wait for it to finish
const run = await client.actor("epctex/playstation-store-scraper").call(input);
// Fetch and print actor results from the run's dataset (if any)
console.log('Results from dataset');
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
})();
from RealdataAPI_client import RealdataAPIClient
# Initialize the RealdataAPIClient with your API token
client = RealdataAPIClient("<YOUR_API_TOKEN>")
# Prepare the actor input
run_input = {
"search": "food game",
"startUrls": [
"https://store.playstation.com/en-tr/product/EP4389-CUSA17440_00-DCLGAMEEU0000000",
"https://store.playstation.com/en-tr/search/game",
"https://store.playstation.com/en-tr/concept/10000886",
"https://store.playstation.com/en-tr/category/d42c9dc6-5516-4a34-a511-c09894266d98/1",
],
"maxItems": 20,
"endPage": 1,
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": { "useRealdataAPIProxy": True },
}
# Run the actor and wait for it to finish
run = client.actor("epctex/playstation-store-scraper").call(run_input=run_input)
# Fetch and print actor results from the run's dataset (if there are any)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# Set API token
API_TOKEN=<YOUR_API_TOKEN>
# Prepare actor input
cat > input.json <<'EOF'
{
"search": "food game",
"startUrls": [
"https://store.playstation.com/en-tr/product/EP4389-CUSA17440_00-DCLGAMEEU0000000",
"https://store.playstation.com/en-tr/search/game",
"https://store.playstation.com/en-tr/concept/10000886",
"https://store.playstation.com/en-tr/category/d42c9dc6-5516-4a34-a511-c09894266d98/1"
],
"maxItems": 20,
"endPage": 1,
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useRealdataAPIProxy": true
}
}
EOF
# Run the actor
curl "https://api.RealdataAPI.com/v2/acts/epctex~playstation-store-scraper/runs?token=$API_TOKEN" /
-X POST /
-d @input.json /
-H 'Content-Type: application/json'
country
Optional Enum
Choose the preferred country.
es-ar string,
en-bg string,
fr-ca string,
en-ca string ,
de-de string,
en-us string,
zh-hant-hk string,
en-in string,
sl-si string
search
Optional String
It is an input field to enter the keyword you want to search on the PlayStation Store.
startUrls
Optional Array
Adding the starting URLs for lists or page details is an input parameter.
maxItems
Optional Integer
Enter the maximum listing item count with all as a default value to get the required output.
endPage
Optional Integer
It is an input field to add the last page number to end the list with no, by default, end page. You can apply it to every start URL and search request separately.
extendOutputFunction
Optional String
It takes an argument based on the JQuery handle ($), reflects the output data, and merges it with the default result.
customMapFunction
Optional String
It takes every object as an argument, reflects the output data, and maps it.
proxy
Optional Object
Choose a proxy server to support the successful scraper execution.
{
"country": "en-us",
"search": "food game",
"startUrls": [
"https://store.playstation.com/en-tr/product/EP4389-CUSA17440_00-DCLGAMEEU0000000",
"https://store.playstation.com/en-tr/search/game",
"https://store.playstation.com/en-tr/concept/10000886",
"https://store.playstation.com/en-tr/category/d42c9dc6-5516-4a34-a511-c09894266d98/1"
],
"maxItems": 20,
"endPage": 1,
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
"proxy": {
"useRealdataAPIProxy": true
}
}
Disclaimer : Real Data API only extracts publicly available data while maintaining a strict policy against collecting any personal or identity-related information.
