The steps we go through to get results...
- Scraping: We scrape the main (home) page of the site and any link(s) (pages) we find on this page.
- We check each page to make sure it has one of our keyword in it.
- All pages with keywords are added to a cumulative file called "pages".
- The "pages" file is examined and any line(s) with less than 4 words is removed.
- The number of pages we scraped are counted and updated in the "Scraped2" column in the sites table.
- Ask ChatGBT
- First we upload the cumulative file "pages".
- Then we "ask" the first 5 questions (in one call) and get the response in a JSON format.
- Question #6 is asked and answered separately.
- Cost for the first 1000 is:
- Cost: 0.86
- Hours: 5.50
- Disk MB: 334.00
- Estimates all:
- Cost: 160.56
- Days: 42.79
- Disk GB: 62.36
- Final Deterimations
- This is handled by Peter at the moment.