Scrape All Pages - Page scraper, then go state by state to send update emails - for any website Version 2020-11-07

This will extract the names, addresses, websites urls, email address, phone numbers and content for each bulleted entry on the page

How to use the form: run  1) this file from pickyourown.org 2), It saves files to the _private dir on server. 3) The files must exist (blank) first time Each time you run the file, it appends more to the csv.

Which website?

PickYourOwn
Easter
Farm Markets
PumpkinPatchesAndMore
ChristmasTrees
Other (enter details below)

URL of the page to convert:. (can ignore this)

   

Generate (T)est emails (S)end the emails or  (N)othing          

Test - scrapes and shows results, but does not send emails or store anything
Send - sends emails to email address on the page, and a copy to "emailupdates@domain.org  (Use test page to test the emails first)
Create a CSV file and saves it to the _private dir on either the server or localhost, depemding where you are running it from (no emails are sent, just a summary emailed to myself at "emailupdates@domain.org

The rest of these choices (below) are automatic, based on the two choices above, unless you select O (other).

Test page is TE.htm, TE.php, TEfarmmarket.php, etc. with subpages TEtest1.htm, TEtest2.htm, or TEtest1.php, etc. The email addresses all go back to me.


What state is this for: All Alabama to Maine Mass to SC SD to WY
Test page
Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
DC
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Massachusetts
Maryland
Michigan
Minnesota
Mississippi
Montana
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming


You can ignore settings below for now.  Settings are hard coded.

What string defines the beginning of the listings:

What database to use:

What string defines the end of the listings:

What is the tablename:

What string defines separates each listing: (only Other category)

 

See PYO for master code. Note: "explode" requires the content be bracketed by the delimiter, so if there is only ONE county
on a page, it won't work.  Solution is add an h3 tag at the end of the listings, with innocuous content like
"<h3>For more information see the top menu bar</h3>"

Other required files: (last modified 2020-11-06

scrape-all-results.php


Benivia, LLC
Copyright © 2019 Benivia, LLC. All rights reserved.
Revised: 06/15/24