Nrthugu

URL: https://myanimelist.net/anime/236/Es_Otherwise

I trying to scrape the following content in URL:

I tried :

for i in response.css('span[class = dark_text]') :

    i.xpath('/following-sibling::text()')

or that current XPath who's don't work or I missed something...

aired_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[11]/text()')



producer_xpath = response.xpath("//*[@id='content']/table/tbody/tr/td[1]/div/div[12]/span/a/@href/text()")

licensor_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[13]/a/text()')

studio_xpath response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[14]/a/@href/title/text()')

studio_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[17]/text()')

str_rating_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[18]/text()')

ranked_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[20]/span/text()')

japanese_title_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[7]/text()')

source_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[15]/text()')

genre_xpath = [response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[16]/a[{0}]'.format(i)) for i in range(1,4)]

genre_xpath_v2 = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[16]/a/@href/text()')

number_of_users_rated_anime_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[19]/span[3]/text()')

popularity_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[21]/span/text()')

members_xpath = response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[22]/span/text()')

favorite_xpath =  response.xpath('//*[@id="content"]/table/tbody/tr/td[1]/div/div[23]/span/text()')

but I figured out that some text are out of a span class, so I would like to get that text out of span with a css/XPath formula.

If you are only trying to scrap the information that you mentioned in the image you can just make use of

response.xpath('//div[@class="space-it"]//text()').extract()

Or i am unable to understand your question properly.

it simpler to just loop through div inside the table

foundH2 = False

response =  Selector(text=htmlString).xpath('//*[@id="content"]/table/tr/td[1]/div/*')



for resp in response:

  tagName = resp.xpath('name()').extract_first()

  if 'h2' == tagName:

    foundH2 = True

  if foundH2:

    # start adding 'info' after <h2>Alternative Titles</h2> found

    info = None

    if 'div' == tagName:

      for item in resp.xpath('.//text()').extract():

        if 'googletag.' in item: break

        item = item.strip()

        if item and item != ',':

          info = info + " " + item if info else item

      if info:

        print info

just my opinion, beautifulSoup is faster and better than scrapy.