Jump to content

  • Log in with Facebook Log In with Google      Sign In   
  • Create Account

Hey Guest, Welcome to Traffic Planet!

Sign up today in order to gain access to a vast range of features including the ability to create new topics, send private messages, Facebook & Twitter integration and MUCH more!

  • RSS Feed
  • Citations without links (automation help needed)

    citations link building

    • Please log in to reply
    7 replies to this topic

    #1 morellowed

    morellowed

      New Member

    • New Member
    • posts 4
    • Joined: 23-December 16
      Reputation: 0
    • LocationBirmingham, UK

    Posted 23 December 2016 - 04:53 PM

    Hi,

     

    This is my first post here, so hello!

     

    I hope to be able to learn a lot and to contribute too.

     

    I have recently read a little about scrapers and I am wondering if the following is possible:

     

    I would like to paste in a list of URLs and have the scraper go to each page and look in the source code for a domain that I ask it to look for. I would then like to have the results in a spreadsheet.

     

    The list of URLs would be pages where a certain business is mentioned and where there may/ may not be a link.

     

    I can then follow up mentions of a business that may not have a link to see if I can get one.

     

    Does anyone know how this could be done using any software?

     

    Is it even a scraper that I need?

     

    Any help appreciated.

     

    M



    #2 _Richard

    _Richard

      Forum Moderator

    • Forum Support
    • posts 6,031
    • Joined: 27-September 11
      Reputation: 2,156
    • LocationNW Arkansas

    Posted 23 December 2016 - 07:20 PM

    All you want back is a list of the urls from your initial list that contain the text string (in this case a business name) on the page?

     

    If so that should be pretty easy to do. If you have ScrapeBox you should be able to use it. If not any scraper should be able to do that as you are not asking for much info.

     

    Also if this is a one time thing or a test instead of investing in the software I  bet you can find a fiverr gig that will do it for you.


    Sometimes I have a link in my signature to a product. If I do assume it is an affiliate link and I might make a couple of bucks off it...................................


    #3 morellowed

    morellowed

      New Member

    • New Member
    • posts 4
    • Joined: 23-December 16
      Reputation: 0
    • LocationBirmingham, UK

    Posted 23 December 2016 - 08:56 PM

    All you want back is a list of the urls from your initial list that contain the text string (in this case a business name) on the page?

    If so that should be pretty easy to do. If you have ScrapeBox you should be able to use it. If not any scraper should be able to do that as you are not asking for much info.

    Also if this is a one time thing or a test instead of investing in the software I bet you can find a fiverr gig that will do it for you.

    I have a list of URLS which mention a business. I now want to check if each URL has a link to the business.

    If The URL does not link to the business, I want to contact them to get a link.

    I want to do this on an ongoing basis.

    How do I get started with a scraper?

    M

    Edited by morellowed, 23 December 2016 - 09:00 PM.


    #4 _Richard

    _Richard

      Forum Moderator

    • Forum Support
    • posts 6,031
    • Joined: 27-September 11
      Reputation: 2,156
    • LocationNW Arkansas

    Posted 24 December 2016 - 02:36 AM

    OK I had it backwards but you are still taking a list of URLs and splitting it into two list using just one yes/no criteria which is a pretty simple task.

     

    You should have no problem doing this with scrapebox if you have it. If not it may still be the cheapest option. It has been awhile but I am pretty sure I have used it for similar work.

     

    I did a quick search for dedicated scraping tools and they all looked fairly expensive and very much overkill for what you want to do. If you dig deeper you may find one more reasonably priced.

     

    If you know anyone with some coding skills they might be able to create a dedicated bot for that. Fairly simple task it should not take more than a few hours tops.


    Sometimes I have a link in my signature to a product. If I do assume it is an affiliate link and I might make a couple of bucks off it...................................


    #5 morellowed

    morellowed

      New Member

    • New Member
    • posts 4
    • Joined: 23-December 16
      Reputation: 0
    • LocationBirmingham, UK

    Posted 24 December 2016 - 11:15 AM

    Thanks. I was hoping there would be a free tool for this task, such as a scraper add on for Chrome: http://chrome.google.com/webstore/detail/scraper/mbigbapnjcgaffohmbkdlecaccepngjd?hl=en

    #6 _Richard

    _Richard

      Forum Moderator

    • Forum Support
    • posts 6,031
    • Joined: 27-September 11
      Reputation: 2,156
    • LocationNW Arkansas

    Posted 24 December 2016 - 01:38 PM

    Thanks. I was hoping there would be a free tool for this task, such as a scraper add on for Chrome: http://chrome.google.com/webstore/detail/scraper/mbigbapnjcgaffohmbkdlecaccepngjd?hl=en

     

    There may well be it is just going to be a matter of finding one that works for you. Just  a matter of doing searches and looking at all the choices.

     

    Doing a quick search there are some free choices out there. 

     

    https://www.quora.com/What-are-some-good-free-web-scrapers-scraping-techniques

    http://nocodewebscraping.com/best-free-web-scraping-tools-for-non-programmers/

     

    Another chrome extension (actually not something I had even thought of)

    https://chrome.google.com/webstore/detail/web-scraper/jnhgnonknehpejjnehehllkliplmbmhn


    Sometimes I have a link in my signature to a product. If I do assume it is an affiliate link and I might make a couple of bucks off it...................................


    #7 morellowed

    morellowed

      New Member

    • New Member
    • posts 4
    • Joined: 23-December 16
      Reputation: 0
    • LocationBirmingham, UK

    Posted 24 December 2016 - 04:23 PM

    There may well be it is just going to be a matter of finding one that works for you. Just  a matter of doing searches and looking at all the choices.

     

    Doing a quick search there are some free choices out there. 

     

    https://www.quora.com/What-are-some-good-free-web-scrapers-scraping-techniques

    http://nocodewebscraping.com/best-free-web-scraping-tools-for-non-programmers/

     

    Another chrome extension (actually not something I had even thought of)

    https://chrome.google.com/webstore/detail/web-scraper/jnhgnonknehpejjnehehllkliplmbmhn

     

    Thanks for the links - I'll check these out.

     

    I manged to find these guys: https://www.datascraping.co/

     

    There is a Chrome extension and you can manage things in browser and there is also bit of software to download. Their support have said what I want is possible and have offered to do a demo.


    Edited by morellowed, 24 December 2016 - 04:24 PM.


    #8 Broland

    Broland

      New Member

    • New Member
    • posts 4
    • Joined: 26-November 12
      Reputation: 1

    Posted 27 December 2016 - 12:20 AM

    There are free scripts out there for this if you dig. I've seen some in various languages on Github for example. Try searches with things like: check if URL exists.

     

    I actually have one I used to use for the same purpose. Happy to send you the script if you want to shoot me a PM with your email. You'll need to host it locally or put it up on hosting. The functionality of what you are after is pretty basic, the more detailed stuff comes down to the details such as how it is hosted, whether you are uploading a file of URL's each time and downloading a return file, or want it web hosting with form input and outputs, scaling it do work faster and handle more etc. I could recommend some coders too if needed.

     

    Out of curiousity, how are you getting your list of URL's with mentions in the first place? Scraping Google or something? If you have something already set up for that, modifying/adding to that with additional steps might be your best end result for saving time on the entire task.


    Superior Local SEO, Business Branding and Consulting Services at DirectoryBug.com






    Similar Topics Collapse


    0 user(s) are reading this topic

    0 members, 0 guests, 0 anonymous users