l i n u x - u s e r s - g r o u p - o f - d a v i s
Next Meeting:
July 7: Social gathering
Next Installfest:
Latest News:
Jun. 14: June LUGOD meeting cancelled
Page last updated:
2006 Sep 04 18:48

The following is an archive of a post made to our 'vox-tech mailing list' by one of its subscribers.

Report this post as spam:

(Enter your email address)
Re: [vox-tech] PHP / CURL
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [vox-tech] PHP / CURL

On Sep 1, 2006, at 10:35 AM, serendipitu serendipitu wrote:

I need to READ some data from that page without manually loging in every 24 hours.
PHP/curl makes this pretty easy (depending on how much energy the site developers have put into trying to prevent screen-scraping). Also, any language that has a curl implementation can also do this (PERL is one that comes to mind).

You need a pretty strong understanding of PHP and a basic understudying of how HTTP works. You'll need a webserver that runs PHP or a local machine with the PHP command line interface installed. Then you'll need a script. That script will take a series of steps that each represent a login, a link click, a form submission, or some kind of user interaction with a website.

The process basically works like this:

First you call curl_init() to get things started.

You need to call curl_setopt() any number of times to define what type of call you're going to make (in this case a series of HTTP transactions). These curl_setopt() calls are very similar to the command line switches you'd throw at the command line version of curl.

Then you finish up with a curl_exec() and a curl_close().

It took me a lot of ready and trial and error to figure this all out. I'd start here: http://www.php.net/manual/en/ref.curl.php

Every site is different, and it's difficult to tell you what to do without having a half.com account.

vox-tech mailing list

LUGOD Group on LinkedIn
Sign up for LUGOD event announcements
Your email address:
LUGOD Group on Facebook
'Like' LUGOD on Facebook:

Hosting provided by:
Sunset Systems
Sunset Systems offers preconfigured Linux systems, remote system administration and custom software development.

LUGOD: Linux Users' Group of Davis
PO Box 2082, Davis, CA 95617
Contact Us

LUGOD is a 501(c)7 non-profit organization
based in Davis, California
and serving the Sacramento area.
"Linux" is a trademark of Linus Torvalds.

Sponsored in part by:
Sunset Systems
Who graciously hosts our website & mailing lists!