Python - Crawling images from json file

Python – Crawling images from json file

Analysis

url: https://www.luscious.net/albums/cassie-laine_348936/

load images with ajax. It returns json file

url: https://api.luscious.net/graphql/nobatch/?operationName=AlbumListOwnPictures&query=+query+AlbumListOwnPictures%28%24input%3A+PictureListInput%21%29+%7B+picture+%7B+list%28input%3A+%24input%29+%7B+info+%7B+...FacetCollectionInfo+%7D+items+%7B+...PictureStandardWithoutAlbum+%7D+%7D+%7D+%7D+fragment+FacetCollectionInfo+on+FacetCollectionInfo+%7B+page+has_next_page+has_previous_page+total_items+total_pages+items_per_page+url_complete+%7D+fragment+PictureStandardWithoutAlbum+on+Picture+%7B+__typename+id+title+description+created+like_status+number_of_comments+number_of_favorites+moderation_status+width+height+resolution+aspect_ratio+url_to_original+url_to_video+is_animated+position+tags+%7B+category+text+url+%7D+permissions+url+thumbnails+%7B+width+height+size+url+%7D+%7D+&variables=%7B%22input%22%3A%7B%22filters%22%3A%5B%7B%22name%22%3A%22album_id%22%2C%22value%22%3A%22348936%22%7D%5D%2C%22display%22%3A%22rating_all_time%22%2C%22page%22%3A1%7D%7D

Python - Crawling images from json file

Coding

json structure below. We need the highest resolution images.

Python - Crawling images from json file

import requests
import json
from urllib.request import urlretrieve

def download(url):
   header = { "User_Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36" }
   html = requests.get(url, headers=header).text  #return json file
   jobj = json.loads(html)

   items = jobj['data']['picture']['list']['items']
   for item in items:
      img = th['url_to_original']
      filename = img[img.rfind("/") + 1:]
      filepath = 'd:/img/'+filename
      print(filepath)
      urlretrieve(img, filepath)

url = "https://api.luscious.net/graphql/nobatch/?operationName=AlbumListOwnPictures&query=+query+AlbumListOwnPictures%28%24input%3A+PictureListInput%21%29+%7B+picture+%7B+list%28input%3A+%24input%29+%7B+info+%7B+...FacetCollectionInfo+%7D+items+%7B+...PictureStandardWithoutAlbum+%7D+%7D+%7D+%7D+fragment+FacetCollectionInfo+on+FacetCollectionInfo+%7B+page+has_next_page+has_previous_page+total_items+total_pages+items_per_page+url_complete+%7D+fragment+PictureStandardWithoutAlbum+on+Picture+%7B+__typename+id+title+description+created+like_status+number_of_comments+number_of_favorites+moderation_status+width+height+resolution+aspect_ratio+url_to_original+url_to_video+is_animated+position+tags+%7B+category+text+url+%7D+permissions+url+thumbnails+%7B+width+height+size+url+%7D+%7D+&variables=%7B%22input%22%3A%7B%22filters%22%3A%5B%7B%22name%22%3A%22album_id%22%2C%22value%22%3A%22348936%22%7D%5D%2C%22display%22%3A%22rating_all_time%22%2C%22page%22%3A1%7D%7D"
download(url)
print('done!')

Python - Crawling images from json file

Result

Python - Crawling images from json file

232 total views, 1 views today

Author: Albert

Leave a Reply