Step 3 Import CSV to MySQL, Version 2024-06-20

This will extract the names, addresses, websites urls, email address, phone numbers and content for each bulleted entry on the page

Do this AFTER scrape-all-form scrapes all pages

WARNINGS:

  1.  Apostrophes (') must be escaped (\).
  2. After you scrape all the pages with scrape-all-form-2022-q.php you will need to download the scraper csv file, delete the first (blank) line of the CSV save as a CSV
  3. You must find and replace all ’ with ' and - with - (you can do this in the scrapertable CSV (for PYO, that is ScraperResultsCSV-pyo.csv  in _private dir)
  4. Re-upload the CSV file to the correct directory

 

How to use the form: run  1) this file from pickyourown.org 2), It saves files to the _private dir on server. 3) The files must exist (blank) first time Each time you run the file, it appends more to the csv.

Which website?

PickYourOwn - uses _private/ScraperResultsCSV-pyo.csv
Easter - ScraperResultsCSV-east.csv
Farm Markets - ScraperResultsCSV-lfm.csv
USDA Farm Markets - ScraperResultsCSV-usda.csv
PumpkinPatchesAndMore - ScraperResultsCSV-pump.csv
Christmas Trees - ScraperResultsCSV-xmt.csv
Other (enter details below)

Which CSV file (for OTHER selection above only):

for now, the structure of the CSV files must be as follow, so add anything else after 21, catchalls:

0         1             2         3         4         5        6     7     8     9         10         11                 12             13     14             15         16             17     18         19             20         21            22            GPS 1      GPS 2
name, closed, email, phone, addr1, addr2 city, state, zip, county, redirect, farmwebsite, facebook, crops, whichweb, onpage, pagedesc, open, payment, organic, directions, catchall      uniquieID  GPS1        GPS2

NOTHING below has any impavt

Generate (T)est emails (S)end the emails or  (N)othing          

Test - scrapes and shows results, but does not send emails or store anything
Send - sends emails to email address on the page, and a copy to emailupdates@domain.org (Use test page to test the emails first)
Create a CSV file and saves it to the _private dir on either the server or localhost, depending where you are running it from (no emails are sent, just a summary emailed to myself at "emailupdates@domain.org

Exclude recent listings (current year) (Either ADDED or UPDATED matches date range) (THIS IS NOT USED NOR WORKING YET)

0. No filter
1. Jan- March 2024
2. Any month in 2024
3. Confirmed (span class="pink" for Easter) (Note, doesn't work, span gets stripped out before the test is applied)

The rest of these choices (below) are automatic, based on the two choices above, unless you select O (other).

Test page is TE.htm, TE.php, TEfarmmarket.php, etc. with subpages TEtest1.htm, TEtest2.htm, or TEtest1.php, etc. The email addresses all go back to me.

Note: State or country region pages with no subpages require an exception on line 85 of results. or else you will get a "Warning: file_get_contents" error

Select ONE Country: US, Canada, UK, AU, NZ,


What state is this for: All Alabama to Maine Mass to SC SD to WY
Test page
UK Test page
SINGLE PAGE - enter url below

Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
DC
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Massachusetts
Maryland
Michigan
Minnesota
Mississippi
Montana
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming
Canada
Alberta
BC
Manitoba
NewBrunswick
NewFoundland
Nova Scotia
Ontario
PEI
Quebec
Saskatchewan
Yukon
Britain
London
Southern East Anglia
Northern East Anglia
Berks, Bucks,Ox
Hampshire
Kent
Sussex
East Midlands
West Midlands
Northeast England
Northwest England
Cornwall and Devon
Southwest
Yorkshire
Scotland
Wales
Northern Ireland
Australia
New South Wales
Northern territory
Queensland
South Australia
Tasmania
Victoria
Western Australia
New Zealand
New Zealand

aaa

URL of a SINGLE PAGE  to run:. (only use with the SINGLE PAGE checkbox above)

   


You can ignore settings below for now.  Settings are hard coded.

What string defines the beginning of the listings:

What database to use:

What string defines the end of the listings:

What is the tablename:

What string defines separates each listing: (only Other category)

 

See PYO for master code. Note: "explode" requires the content be bracketed by the delimiter, so if there is only ONE county
on a page, it won't work.  Solution is add an h3 tag at the end of the listings, with innocuous content, see notes below

Other required files: (last modified 2022-08-23)

scrape-all-results-2022-i.php (or more recent equivalent)

Notes:

On region pages on PYO, if you have other bullet/<li> links below the links to the subpages, you must separate them with the endlistings separator.

For PYO, that is:

<h3><a name="farmmarkets">Other Local Farm Products (Honey, Horses, Milk, Meat, Eggs, Etc.)</a>
<br>(NOT pick-your-own, unless they are also listed above)</h3>

Example FLnorth.htm

Version changes

2022-b adds second street addr2


Benivia, LLC
Copyright © 2019 Benivia, LLC. All rights reserved.
Revised: 05/07/25