ScriptWritingPeople

Questions

Ethically - you're the theologian ;) Legally - grey area. The whole business of internet publishing is fraught with uncertainty about precisely what implicit licence you grant others to copy your work for the purpose of viewing it.

Have you tried looking at Google's cached copy? --TheInquisitor

Sorry, ignore that - I hadn't realised it wasn't the top hit. I am rather curious as to how their spider had a subscription to the journal, though.  --TI

(PeterTaylor) It's quite common to allow Google privileges which ordinary people don't have. This leads me to wonder whether the article might be obtained by setting the User-Agent header to Googlebot. My attempts to use telnet to get their webpage are unsuccessful, though - maybe they don't like HTTP/1.0, and I can't be bothered to work out a complete HTTP/1.1 request.

That would be theft, surely - as you're obtaining it by illicit means ( deception )... -- Xarak

(PeterTaylor) It's certainly not [theft]. I'm not sure whether or not there's deception involved. It's well-known that some browsers allow the user to fake their User-Agent in order to get it to display pages which wouldn't otherwise display (e.g. because their author thinks that IE is the only browser that can display it).

I would imagine they check for the alleged bot coming from Google's IP space. I know *I* would. - MoonShadow

(Untried) information I have from locked lj posts suggests that in the general case they don't... -- Senji

"Since you can only get a small amount of additional text" - is this true? The impression I got was that the entire original was in the database, and could be retrieved by automated piecing together of search results. Hm. I think a proof-of-concept might be in order.. - MoonShadow

sl236@debian:~/work/perl$ perl retrieve.pl 'of Derrida Rorty or Fish compelling will'
of Derrida Rorty or Fish compelling will find that Vanhoozer inade quately represents the views of their favorite tending instead toward a melange of stock characterizations of postmodernism 
sl236@debian:~/work/perl$ 

#! /usr/local/bin/perl -w

$SIG{CHLD}='IGNORE';
my $max = 7;
my $lq = '';

sub fetch
{
  my($q) = @_;
  $q=~s/[^a-zA-Z0-9_]/ /g;
  my $oq = $q;
  $q=~s/ +/+/g;
  $q = $lq . $q;
  $lq = $q;
  $oq =~ s/ +/ /g;

  {
   my @q = split(/\+/,$q);
   if($#q > $max)
   {
     splice @q, 0, $#q - $max;
     $q = join('+',@q);
   }
  }

  open KID_PS, "-|", "links", '-dump', 'http://www.google.co.uk/search?q=%22' . 
$q . '%22';
  my $result = join ('', <KID_PS>);
  close KID_PS;
  $result=~s/-/ /g;
  $result=~s/\.\.\./-/g;
  $result=~s/[^a-zA-Z0-9_-]/ /g;
  $result=~s/ +/ /g;
  if($result =~ /\-.+?$oq(.+?)\-/)
  {
    return $1;
  }
  return undef;
}


my $z = $ARGV[0];
print $z;
while(
     defined( $z = fetch ($z) )
  )
{
  print $z;
};
print "\n";