#!/usr/bin/perl
#
$bapropos = q`
mira(1r) - text-only, command-line oriented web browser
`;

$esc = "\033";
$g_bold = $esc . "[1m";
$g_normal = $esc . "[0m"; $zn = "\005";

$b = $g_bold; $n = $g_normal;
$cmdhelp = "
MIRA (Munafo's Internet Research Assistant) is a simple web browser
created to satisfy the following objectives (listed by priority):

- When you quit or restart the browser after a crash, it retains the
  history ('back' command's URL list) you had before the quit/crash

- Keep a permanent record of where you have been and the text you have
  read while searching the Internet

- Allow viewing of old web pages (and old versions of current web
  pages) even after the original server no longer has them

- Allow the user to easily search through all this old saved text for
  one or more words or a phrase, etc.

- Retain simplicity by remaining entirely text-only (like $b"."lynx$n) and
  rely on external programs (like $b"."xv$n and $b"."arena$n) to view graphics
  and graphics-intensive pages
";

$help = qq`

NAME

   MIRA - text-only, command-line oriented web browser

DESCRIPTION

$cmdhelp
For more perl goodness, go to mrob.com/pub/perl

SEE ALSO

  bget           - Download or stream data from a URL to a file or stdout
  strip-wayback  - Remove archive.org headers and links from HTML

`;

$unused_header = q`

Revision History:
 19980706 Discover nested forms in Hotbot results, and trying to
submit something from the outer form that occurs after the end of the
inner form doesn't work. Fix by using shift and unshift to keep a
stack of form numbers.
 19980723 Add 'gsu' command to make global history searches easier!
 19980916 Add routines to save and restore the "local history stack".
 (period of many undocumented changes)
 19981218 Begin recording dates in global log (g_log) file
 19990126 Over the last few days I have added color-coding to show
what links have been visited, and of those that have not, which are on
the same host as the current page. This aids in the (recently more
frequent) practice of manually loading everything about a given
subject just so I can have the text in the archive.
 19990127 add support for viewing PDF's through gv, but gv doesn't
seem to display the PDF's properly.
 19990201 #glr# command works. (#ghr# is still pending). Fix bugs in
#gsc# and add default command to #gsc# making it easy to continue the
search to older pages.
 19990208 Form submissions now include values of checked "radio"
items, and recognize numerical values of "select" items; this makes
DejaNews queries work properly.
 19990209 #f# command now supports a numeric argument; added #sf#
command; #bm# now prints the title and URL it's adding
 19990211 Now you can type #rf# to view form fields after you've
filled them in; #m# and #vs# commands include similar changes.
 19990212 Now you can type "a .." from a URL that ends in "/" and it
will go up one directory. This complments the use of "a " from URLs
that don't end in "/".
 19990216 Added #ga# command allowing jump to an anchor based on text
in the anchor's label. Often useful on pages returned by search
engines, where anchors like "Next" occur with an unpredictable number
but a predictable name.
 19990219 Hotbot has now set up their links to be queries to HotBot,
which return a page containing a redirected URL. Modify MIRA to handle
this by retroactively changing its internal URLs (in anc_base, stack,
global history and log) and copying the already-loaded data into
another place in the cache.
 19990223 Fix a bug that made DejaNews queries not get cached
 19990224 Yet another bug that made redirected pages (like HotBot
hits) not get cached. This time it's fixed a lot better, I found that
there was some confusion as to where the extra cache copy should be
created and decided to put it completely in load_n_cache.
 19990301 formatting chain now generates an output file containing
plain ASCII output; #p# now has an option to save in plain ASCII
format. #p#'s (S) and (A) options append if file already exists.
 19990302 #p# command switches color to magenta.
 19990303 Fix some bugs in newline formatting to plain ASCII output.
 19990304 Add URL and date stamp to beginning of plain ASCII output
file. #gsc# now takes a quoted argument to allow searching for strings
containing spaces.
 19990308 add_anchor now gleans hrefs from within query hrefs,
essentially restoring direct links to HotBot query results pages (see
19990219). This allows the user to see if the link has been visited,
very useful when a new search returns links visited through a previous
search. Also, direct links are often faster and more reliable. Make
AREA anchors have colors just like normal anchors.
 19990310 AREA labels now show part of their href text to make them
less ambiguous. Fixed bug with #f# in GHR mode.
 19990312 Found a page that ends anchors with </td> instead of </a>.
 19990325 add_anchor1 now recognizes "javascript:openWindow" in URLs.
<script> is now handled even if <html> hasn't been encountered yet
(happens on MSNBC's pages). Add several &#xxx character mappings with
leading zeros (as in &#0143 rather than &#143) which have just started
appearing today.
 19990326 Fix bug that made it sometimes ignore timed refresh
directives. Recognize and print <meta> summary and keywords text.
 19990330 Print all unrecognized <meta .. content> text. Expand
add_anchor's gleaning to include ftp hrefs within results of queries
to mp3.lycos.com.
 19990408 Minor changes to mailto: handling.
 19990429 </ul> now causes a line break. Added special <hr> handling
to proc10 in attempted fix of <hr> glitches when web page uses
indentation.
 19990504 It now creates its directories if they're not there, or
complains and exits.
 19990513 Add parameter to #sa#, which prints only those anchors whose
label matches a pattern.
 19990514 Changes to support KWM, kvt, and Netscape.
 19990525 Add a frame only if its URL is unique (prevents infinite
loop from frames that contain themselves as subframes)
 19990526 Add #dl im# command to make downloading images easier.
 19990528 Add #mc# command.
 19990601 Add #bh# command.
 19990607 Remove the "input1" label and all the goto's; replace them
with the skip_to_input flag. This fixes a mysterious error message
from perl "target of goto is too deeply nested". Insert many quotes in
comments at the ends of lines containing =~ patterns, to satisfy
Emacs' keyword coloring.
 19990608 Add #ro# command and display of "redirected from" URL if
applicable.
 19990622 Add '-c' command line argument, which just loads a page and
puts its data into the cache. Fix bug that's been around since around
Feb 24th when I made it deal with redirected pages -- for
non-redirected pages it was recording the same URL twice in a row into
g_log -- very wasteful.
          Write the #md1# script, which monitors #netscape#'s index
files and passes each visited URL to #mira#. This allows me to keep
the #mira# cache up to date with pages I've visited with the external
viewer. To support this, I add the $goodload and $justcache flags and
associated behavior. It adds URLs to the global log and history only
if they turn out to be good URLs (many of the URLs found in the
Netscape files are truncated or have garbage characters at the end).
 19990623 Add the $isgif flag; pages are formatted and cached only if
they are not GIF files. This is to avoid caching lots of ad banners
sent to #mira# by #md1#.
 19990628 Write #str1.c#, which prints strings from the netscape index
only if they're followed by a null character. This dramatically
improves #md1# by cutting out most (or perhaps all?) of the "garbage"
strings.
 19990629 Incorporate the #md1# (now called #mirad#) script and the
#str1# source code into #mira#, and make #mira# install them on
startup. #mirad# still needs to be manually launched by the user. Make
#str1# do the checking for "http://" and move the GIF/JPG/etc check
earlier in the pipeline, both to improve efficiency.
 19990725 Discover that the Netscape history file is sometimes called
"history.dat" and sometimes "history.db". Both can be parsed the same
way, but #str1# has to be smart enough to check both. For now, I just
change the name to make it work on machine 5 (the desktop at SH's
apt). Also, add test of $atwork flag to determine whether to set proxy
environment variables.
 19990726 Make #gsc# command smarter. It now takes an initial GHP
parameter, to speed up the common operation of continuing the search
to earlier pages.
 19990825 The cumulative effect of running #mirad# for several months
caused about 130 megabytes of http_spool files to pile up in ~/tmp. I
have added code to unlink the spoolfile in the two normal exit cases
(it will still get left behind on abnormal exits).
 19991001 Add "more" mechanism to proc16, so user can abort really
long listings.
 19991026 Add new scheme for adding compression to caching -- avoids
problem of how to eliminate sometimes-ambiguous URL-to-pathname
algorithms.
 19991102 Add automatic compression on all cache accesses -- so you
can compress the entire cache by doing a "gsc" through it.
 19991104 Change "gzip" to "gzip -f" to eliminate occasional bug where
a copy of #mira# doesn't see a .gz file (due to filesystem being out
of sync) and tries to reload and compress the page from the server.
This usually only happens if you have just visited a page in Netscape,
the page gets cached by #mirad#, and then you visit the page in
#mira#.
          Also, add more tests of "lof_going" to speed 'X' skipping on
pages that don't have newlines in their HTML source.
 19991208 Fix <pp> handling and add support for real non-breaking
spaces. Fix a highlighting bug involving anchors inside bold text. Add
special-case handling of 'javascript' for TV Guide online.
 19991209 Make it not load binary data if you start #mira# in
interactive mode on an image, audio, etc. URL. Move setting of $atwork
to rpmlib.pl.
 19991213 Make proc01 break line into pieces on ^M characters (some
pages didn't parse correctly without doing this)
 19991219 bug: If you do a "gsc", go through a few hits, then use "b"
to back up, it seems to have to reload the pages as if with the "r"
command, even though they were cached.
 20000110 Figure out the bug reported on 19991219: It happens whenever
you do "gb;b". The prefer_live variable was getting set to 1 whenever
visiting a file: URL, putting #mira# permanently in "live" mode (as if
by the #tc# command) Fix handling of prefer_live.
 20000111 Ignore case when matching "http"
 20000112 Add ".png" as a graphic external type
 20000113 $atwork flag is no longer needed, my .bashrc sets the proxy
info.
 20000114 Start using #bwget# instead of #lynx# to load data.
 20000127 Rename $reload to $proxy_reload (but not currently used
because #bwget# doesn't have the option yet). Add parameter to .rlc,
allowing control of $prefer_live (but doesn't seem to be necessary).
Alter #gsc# so that it only adds one page to the f/b stack, namely the
last page seen in any string of consecutive #gsc# commands.
 20000202 #ev# now sends a signal to a running Netscape (if any)
rather than launching a second copy of #netscape#. Fix bug in table
formatting. #vse# no longer displays any spurious Emacs errors.
 20000214 Disable "more" prompting if there are deferred commands.
 20000304 Add suffixes to #mirad#'s ignore-list: .wav, .tgz, .gz, .tar, .z
 20000314 Make #bwget# an explicit path. reload_gh() now reloads
anything whose cachesize is really small (to repair #mirad# errors).
Catch ctrl-C signal during HTTP loads.
 20000316 Write acolor(). #gsc# now highlights the matched text the
first time the page is displayed.
 20000403 "More" prompt now appears every screen_height lines, to
facilitate use of #mira# on the console; and it now accepts any valid
command as input (big major win!)
 20000508 Add description of #dl im# to help (the command has been
here almost a year)
 20000522 Delete control characters from input and detect them in
deferred commands string. This is to minimize the effects of a bug
with #kvt#: something I'm printing is causing #kvt# to send characters
into STDIN, which causes bogus "can't understand command" errors.
 20000903 Encode field names with encode_field()
 20000908 call #bwpost# for form submissions if the method is POST.
 20000928 Subtract 2 more from $screen_height to allow for a blank
line and the "more" prompt
 20001017 Add control-C abort to #gsc# command.
 20001026 Make #mirad# correctly recognize which of the Netscape index
files is active when both are present
 2000110x save command creates header for both ASCII and HTML choices
 20010413 Add !! command to repeat last input line
 20010912 Check for xterm, kvt, or gnome-terminal.
 20011107 Fix a few minor bugs in forms fields handling
 20011110 Better handling of <DIV>
 20011113 Set stty erase mode if ^H or DEL seen in input; Fix bug in
handling of radio items
 20011114 <return> after #f# defaults to #f#; add comments in
command-parsing section
 20020116 Add parsing of <embed src=...> and #dl em# command to
download QuickTime movies. Add #p# option to #p# command (uses my
#print# script, still needs work if it's going to work with #lpr# on
other people's systems).
 20020129 Move automatic stty erase fix to subroutine fixerase.
 20020205 Add detection of /usr/X11R6/bin/display as alternative to
/usr/X11R6/bin/xv
 20020206 Disable indent and exdent for <li> and </li> and added
counterexample in comment; examples of other pages will need to be
found before re-enabling
 20021111 Add $enable_redirects
 20070812 Add (R)HTF output option to #p# command. Fix gzip filename
argument bug
 20071019 Remove hard-wiring of path to #bwget#
 20071021 Improve conversion of <i> and <pre> to RHTF output.
 20080513 Add newlines around <pre> blocks. Include anchor target in
RHTF output (in the process, add a mechanism that can be used for
other special-case RHTF output in the future)
 20080516 Make #im# work on Mac.
 20081013 Save commands now save to the current directory (that the
user was in when launching #mira#). Also, cancel command if user gives
a pathname to an existing directory (happens when they give a null
response).
 20081023 Add a few character translation rules
 20090821 Improve indenting inside <dd> tags
 20090825 Add some UTF-8 conversion, and a fatal error if there is any
unrecognized UTF-8 in the input.
 20091003 Show <a name> labels.
 20091016 Remove some old character conversions that seem to have been
wrong, and temporarily disable UTF-8 decoding (I need to determine how
to detect UTF-8 mode)
 20100228 Convert \xd7 to 'x'.
  Add &install_str1; rename "str1.c" and #str1# binary by adding
"mira-" as prefix; fix printf prototype compile warning.
 20100801 &munge1 now no longer adds '/' to the end of pathnames like
"http://en.wikipedia.org/wiki/Lesotho".
 20110326 Quote output pathname when doing #P# (print/save) and related
functions.
 20111012 Handle <th> tag, handle &#101; and two others for websites that
hide the letters 's', 'e', 'x'.
 20120120 Ignore </mvassign>
 20120412 Add a few character mappings in the &#8192 range. Rename
proc15 to proc16 and add proc14;it now does a better job of
eliminating multiple blank lines.
 20121028 New URL for Copyleft information
 20150414 Restore the UTF8 conversion in remap.chars
 20151111 Ignore some new (HTML5) tags like <nav> and <main>; add a
few UTF-8 sequences
 20160210 Distinguish a few common "<meta ... charset=..." tags
 20160520 Add a few UTF-8 mappings, and ignore a few harmless tags,
Also suppress inline CSS (within <style> tags)
 20160523 Visited links (both normal and bold) are now magenta (and
make new variables g_visited_colour, g_visited_bold to make this
change easier in future). PDFs now open on MacOS systems.
 20220602 Add -h for help

BUGS and TO DO

If you view a PDF, then do 'rf', it re-downloads and views the PDF;
and if you do 'b', it goes to the page prior to the page that linked
to the PDF. To effect "rf" you have to do "b" then "f".

Keep track of style definitions to glean italic and boldface, etc.
from <p class=foo> and <div class=bar> (this entails making the
transport layer re-entrant so I can load .css files when <style> is
encountered)

Compression schemes:

 Stage 1 -- compress in place. Each file in the cache tree is left in
its place, but compressed with ".gz" suffix.

 Stage 2 -- files are collected into archive files (about 1 meg in
size) with hashed indexes to locate where they are.

 existing code -> full pathname of a cache file
                          V
                        hash
                          V
                        index
                          V
                 pathname of archive file
                          V
                 decompression
                          V
                 temp directory of decompressed data
                 needed file moved to its place in normal cache tree

 notes for compression:
  - try to keep archives less than 1 meg, but need to make sure not to
    "jam" if we encounter something huge
  - if current archive + new file uncompressed size > 1 meg, start a new
    archive
  - ignore filenames .*, #*, *#, *~, core
  - compress one each time a command is run to have lower impact on
    responsiveness; it will eventually (months) catch up
  - don't compress anything if "live" cache size area is smaller than a
    certain amount (like 5-10 megs)
  - LRU rule: don't compress something if it's in the most recent N URLs
  - have to make "gsu" a very quick operation in order to implement LRU rule
  - LRU rule difficult in a multi-MIRA environment!

 Stage 3 -- replace URL to pathname conversion with a more direct
URL-to-hash conversion.

 Stage 4 -- checksum page contents and involve the checksum as a step
in the index process, so duplicate pages get stored just once.

 %%% things to do to mirad:
 Konqueror cache: ~/.kde/share/apps/konqueror (filter out \000's,
delimit ends by \377)
 Lots of stuff is getting cached which then results in "GIF output
suppressed" message. Stop cacheing this stuff!

`;


$mirad_text = << 'MIRAD_END';
#!/usr/bin/perl

$help = q`

 mirad -- daemon that watches Netscape index and adds data to MIRA cache

 This script periodically scans through the cache index maintained by
NetScape, and causes MIRA to load all the same pages. This causes the
pages to be loaded into MIRA's cache, making the MIRA cache
automatically include all text you've read with Netscape.

 The job of actually reading the cache index file and extracting the
URLs is done by a separate program, mira-str1.c. mira-str1.c looks for
strings of consecutive ASCII characters terminated by a null
character. This differs from the #strings# command, which doesn't
require the null termination.

`;

sub filter_trunc
{
  local($inf, $outf) = @_;

  open(IX, $inf);
  open(OUT, "> $outf");
  $l2 = "";
  while($l1 = <IX>) {
    chop $l1;
    if ((!($l1 =~ m|/$|)) && (index($l2, $l1) == 0)) {
      # current line is a substring of previous line
    } elsif ($l1 =~ m|\Disk Cache$|) {
      # ignore these too
    } else {
      print OUT "$l1\n";
    }
    $l2 = $l1;
  }
  close IX;
  close OUT;
}

sub path_mtime
{
  local($path) = @_;
  local($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size1,
    $atime,$mtime,$ctime,$blksize,$blocks);

  ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size1,
    $atime,$mtime,$ctime,$blksize,$blocks)
    = stat($path);

  return $mtime;
}

$| = 1;

chdir; $homedir = `pwd`; chop $homedir;

$mddir = "$homedir/mira/mirad";
chdir($mddir);

$nsfile = "history.raw";

# we used to use "$nsdir/cache/index.db";
#
# Find which version of history file exists and link to it.
if (-e $nsfile) {
  unlink $nsfile;
}
$nsdir = "$homedir/.netscape";
$nswhich = 0;
$ns1 = "$nsdir/history.db";
$ns2 = "$nsdir/history.dat";
if (-e $ns1) {
  if (-e $ns2) {
    if (&path_mtime($ns1) > &path_mtime($ns2)) {
      $nswhich = 1;
    } else {
      $nswhich = 2;
    }
  } else {
    $nswhich = 1;
  }
} elsif (-e "$nsdir/history.dat") {
  $nswhich = 2;
}
if ($nswhich == 1) {
  system("ln -s $ns1 $nsfile");
} elsif ($nswhich == 2) {
  system("ln -s $ns2 $nsfile");
} else {
  print "mirad: Can't find Netscape history file.\n";
  exit 1;
}

#   OLD  mira-str1       #str1# binary generates list of URLs
#    |     |
#    |   filter          #mirad# filters out ones that look valid
#    |    gifs
#    |     |             output of this goes into "ix-cur1"
#   OLD   CUR1 (CUR2)    supply file "ix-cur2" to add URLs from
#    |\   /  ..''           another machine
#    | cat ''            "ix-old", "ix-cur1" and "ix-cur2" are
#    |   \                  run through cat | sort | uniq...
#    |  sort+uniq
#    |     |
#   OLD < CUR            ...to produce "ix-cur"
#    |     |
#  filtr filtr           "ix-old" and "ix-cur" are both filtered for
#  trunc trunc           truncated lines
#    |     |
#   OLDF  CURF           to produce "ix-oldf" and "ix-curf"
#     \   /
#      diff              #diff# between these two files
#       |
#      NEW               yields new URLs in "ix-new"

while(1) {
  # Look at Netscape index and extract all nonsecure HTTP URLs
  open(IX, "$homedir/bin/mira-str1 |");
  open(OUT, "> ix-cur1");
  while ($l = <IX>) {
    chop $l;
    $l1 = $l; $l1 =~ s|/$||;
    if ($l =~ m|;|) {
      # Semicolon links are suppressed right now because they cause
      # shell error messages.
    } elsif ( ($l1 =~ m|\.gif$|i)
      || ($l1 =~ m|\.jpeg$|i)
      || ($l1 =~ m|\.jpg$|i)
      || ($l1 =~ m|\.png$|i)
      || ($l1 =~ m|\.rpm$|i)
      || ($l1 =~ m|\.jar$|i)
      || ($l1 =~ m|\.class$|i)
      || ($l1 =~ m|\.pdf$|i)
      || ($l1 =~ m|\.zip$|i)
      || ($l1 =~ m|\.mov$|i)
      || ($l1 =~ m|\.ram$|i)
      || ($l1 =~ m|\.wav$|i)
      || ($l1 =~ m|\.tar$|i)
      || ($l1 =~ m|\.tgz$|i)
      || ($l1 =~ m|\.gz$|i)
      || ($l1 =~ m|\.z$|i)
      || ($l1 =~ m|\.bz2$|i)
    ) {
      # Ignore these file types
    } else {
      print OUT "$l\n";
    }
  }
  close IX;
  close OUT;

  # Make sure there is an ix-old
  if (!(-e "ix-old")) {
    system("echo > ix-old");
  }

  # Combine with database of URLs we've already seen
  # NOTE: eventually we want old and cur to be sorted separately and
  # then merged with sort -m
  system ("cat ix-old ix-cur* | sort -r | uniq > ix-cur");
  # clean up
  unlink "ix-cur1";
  unlink "ix-cur2";

  # Filter out truncated substrings
  &filter_trunc("ix-old", "ix-oldf");
  &filter_trunc("ix-cur", "ix-curf");

  # Now use diff to find out what's new
  open(IX, "diff ix-oldf ix-curf |");
  open(OUT, "> ix-new");
  while ($l = <IX>) {
    chop $l;
    if ($l =~ m|^> |) {
      $l =~ s|^[<>] ||;
      if ($l =~ m|^http:|) {
        print OUT "$l\n";
      }
    }
  }
  close IX;
  close OUT;
  # clean up
  unlink "ix-oldf";
  unlink "ix-curf";

  # Go through the new ones and load them with MIRA
  $gotone = 0;
  open(IX, "ix-new");
  while($l = <IX>) {
    chop $l;

    $l4 = substr($l, -4, 4);
    $l8 = substr($l, -8, 4);
    if ($l8 eq $l4) {
      # don't load these -- they're caused by hash tage overwriting
      # old strings.
    } else {
      # here's one we should actually load
      $gotone = 1;
      $l =~ s|"|\%22|g; # "
      $l =~ s|'|\%27|g;
      print "$l\n";
      system "$homedir/bin/mira -c '$l'";
    }
  }
  close IX;
  # clean up
  unlink "ix-new";

  # This is the only part of the whole program that needs to be atomic
  system ("mv ix-old ix-del; mv ix-cur ix-old");

  unlink "ix-del";

  if ($gotone) {
    print (("_   " x 19) . "\n");
    print ((" \\\\_/" x 19) . "\n");
  }
  sleep 120;
}

MIRAD_END




$str1_c_text = << 'STR1C_END';
/* mira-str1.c */

#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

#define LINEMAX 512

int main(int argc, char **argv)
{
  int in;
  ssize_t len;
  char buf[1024];
  char line[LINEMAX + 2];
  char c1, c2, c3, c4, c5, c6;
  int consec, instring;

  c6 = c5 = c4 = c3 = c2 = c1 = 0;
  consec = instring = 0;

  in = open("history.raw", O_RDONLY);
  while(len = read(in, buf, 1024)) {
    int i, j;
    char c;

    for(i=0; i<len; i++) {
      c = buf[i];

      // update history queue
      c6 = c5; c5 = c4; c4 = c3; c3 = c2; c2 = c1; c1 = c;

      // check for ascii or non-ascii
      if ((c >= 32) && (c < 127)) {
        // ascii
        line[consec] = c;
        if (consec < LINEMAX) {
          consec++;
        }
      } else {
        // non-ascii

        if ((c == 0) && (consec > 8) && (consec < LINEMAX)) {
          // check for "http://"
          if ((line[0] == 'h') && (line[1] == 't') && (line[2] == 't') && 
              (line[3] == 'p') && (line[4] == ':') && (line[5] == '/') && 
              (line[6] == '/'))
          {
            // terminate the string for printing
            line[consec] = 0;

            printf("\%s", line);
            printf("\n");
          }
        }

        consec = 0;
      }
    }
  }
  close(in);
}
STR1C_END

# Returns a nonzero value if this is a Mac OS X system. If possible, the
# value returned is the actual version number (e.g. 10.4)
# The global variables are also set:
#   $g_mkt_ismac        Same as return value of this function
#   $g_mkt_macos_name   Human-readable number and code name
sub ismac
{
  my($mid, $im);

  if ($g_mkt_ismac ne '') {
    return $g_mkt_ismac;
  }

  # Set defaults
  $im = 0;

  $mid = `uname -s`; chomp $mid;
  if ($mid =~ m/Darwin/) {
    $im = 1;
  }
  $g_mkt_ismac = $im;
  return $im;
}

sub init_statics
{
  $q = '"';  # "
  $esc = "\033";
  # This is the system of control characters. Not all are used yet:
  # \001 - CR, LF or CRLF in source
  # \002 - yellow attribute
  # \003 - bold attribute
  # \004 - inverse attribute
  # \005 - normal attribute
  # \006 - nonbreaking or forced blank space (for tables and indentation)
  # \007 - unvisited link, unbold
  # \010 - visited link, unbold
  # \011 (reserved for tab)
  # \012 (reserved for LF)
  # \013 - visited link, boldface
  # \014 - unvisited link, boldface
  # \015 (reserved for CR)
  # \016 - unvisited, same host, unbold
  # \017 - unvisited, same host, bold
  # \020 - bold magenta, for gleaned anchor braces
  # \021 - rule to end of line, ---
  # \022 - rule to end of line, ===
  # \023 - italic start
  # \024 - italic end
  $yellow = $esc . "[0;30;43m"; $zy = "\002";
  $yel7 = $esc . "[0;1;33m";
  $cyan = $esc . "[0;36m";
  $bdcy = $esc . "[0;1;36m";
  $green = $esc . "[0;32m";
  $bdgr = $esc . "[0;1;32m";
  $blue = $esc . "[0;34m";
  $bdbl = $esc . "[0;1;34m";
  $red = $esc . "[0;31m";
  $bdred = $esc . "[0;31;1m";
  $magenta = $esc . "[0;35m";
  $bdmag = $esc . "[0;1;35m"; $zm = "\020";
  $inverse = $esc . "[0;7m"; $zi = "\004";
  $up = $esc . "[A";

  if (0) {
    $esc =
    $yellow =
    $cyan =
    $bdcy =
    $green =
    $bdgr =
    $blue =
    $bdbl =
    $red =
    $bdred =
    $magenta =
    $bdmag =
    $g_bold =
    $inverse =
    $g_normal =
    $up = "";
  }

  $zb[0] = "\005"; # normal
  $zb[1] = "\003"; # bold
  $zb[2] = "\007"; # unvisited normal
  $zb[3] = "\014"; # unvisited bold
  $zb[4] = "\016"; # unvisited normal
  $zb[5] = "\017"; # unvisited bold
  $zb[6] = "\010"; # visited normal
  $zb[7] = "\013"; # visited bold

  $mk_y = $inverse . "y$g_normal";
  $mk_b = $inverse . "b$g_normal";
  $mk_i = $inverse . "i$g_normal";
  $mk_n = $inverse . "n$g_normal";
  $mk_pr2 = "$yellow|$g_normal";

  $ruletoend1 = "\021";
  $ruletoend2 = "\022";

  $nbsp = "\006";

  $g_visited_colour = $magenta;
  $g_visited_bold = $bdmag;
}

# fixerase automatically detects and handles improper setting of
# backspace or delete erase character
sub fixerase
{
  local($l) = @_;

  if ($l =~ m/\010/) {
    # The user's input contains ^H characters.

    # First, perform the intended deletion
    while ($l =~ m/.\010/) {
      $l =~ s/.\010//;
    }

    # Then fix it by setting ^H to be the erase character.
    print "stty erase ^H\n";
    system "stty erase ^H";
  } elsif ($l =~ m/\177/) {
    # The user's input contains delete (^?) characters.

    # First, perform the intended deletion
    while ($l =~ m/.\177/) {
      $l =~ s/.\177//;
    }

    # Then fix it by setting ^? to be the erase character.
    print "stty erase ^?\n";
    system "stty erase ^?";
  }

  return $l;
}

sub munge1
{
  local($url) = @_;
  local($path);

  if ($url =~ m|^http://(.*)$|i) {
    $path = $1;
    if ($path =~ m|/[^\#\?\.\/]+$|i) {
      # leafname has no dot -- append a /
      # This is disabled as of 20100730 because it prevents loading
      # Wikipedia pages like "http://en.wikipedia.org/wiki/Lesotho"
      # $url .= "/";
      # print "munge1 a '$url'\n";
    } elsif ($path =~ m|/|) {
      # nothing
    } else {
      # just a hostname
      $url .= "/";
      # print "munge1 b '$url'\n";
    }
  }
  $url =~ s/ /%20/g;

  return $url;
}

sub munge2
{
  local($url) = @_;

  # perform smart URL mapping
  if ($url =~ m|:/|) {
    # nothing: it's a full URL
    $url = &munge1($url);
  } elsif ($url =~ m|/|) {
    # slash: they gave hostname and path
    $url = "http://" . $url;
    $url = &munge1($url);
  } elsif ($url =~ m|\..*\.|) {
    # two dots: they gave a full hostname
    $url = "http://" . $url . "/";
    # print "munge2 a '$url'\n";
  } elsif ($url =~ m|\.|) {
    # They gave part of the hostname, like olympic.org
    $url = "http://www." . $url . "/";
    # print "munge2 b '$url'\n";
  } elsif ($url eq "") {
    $url = $homepage_url;
  } else {
    # just a company name
    $url = "http://www." . $url . ".com/";
  }

  return $url;
}

# Create and enter a directory in the cache tree
sub cache_mkdir
{
  local($path) = @_;
  local($dir, $subdir);

  if ($path =~ m|^$cachedir/|) {
    $path =~ s|^$cachedir/||;
  }

  $dir = $cachedir;
  chdir($dir);
  while ($path =~ m|^([^/]+)/(.*)$|) {
    $subdir = $1;
    $path = $2;
    $dir .= "/$subdir";
    if (-d $subdir) {
      # don't need to create
      chdir $subdir;
    } elsif (-e $subdir) {
      # it's a file
      print "\n?? trying to create directory over cache file $dir\n";
      exit 0;
    } else {
      mkdir($subdir,0755);
      chdir $subdir;
    }
  }
  if ($path =~ m|\@$|) {
    # last component is a file -- ignore
  } else {
    mkdir($path,0755);
  }
  chdir $path;
} # End of cache.mkdir

sub cache_path
{
  my($url, $full) = @_;
  my($protocol);
  my($host, $hpart);
  my($ttype, $tag, $tid);
  my($rv);

  $url =~ s|\@|\002|g;

  # file: URLs are easy
  if ($url =~ m|^file:(.*)$|) {
    $rv = $1;
    return $rv;
  }

  if ($url =~ m|^([a-zA-Z]+)://(.*)$|) {
    $protocol = $1;
    $url = $2;
    $protocol =~ tr/A-Z/a-z/;
  } else {
    # local file or unknown URL format -- don't cache
    $rv = "";
    return $rv;
  }

  # Extract trailing anchor or query data
  if ($url =~ m|^([^?#]*)([?#])(.*)$|) {
    $url = $1;
    $ttype = $2;
    $tag = $3;

    # Anchors don't affect the cache path at all, so ignore them
    if ($ttype eq "#") {
      $tag = "";
    }
  } else {
    # No tag
    $ttype = "";
    $tag = "";
  }

  # extract host part
  $host = "unknown_host";
  if ($url =~ m|^([^/]+)/(.*)$|) {
    $host = $1;
    $url = $2;

    $host =~ tr/A-Z/a-z/;

    if ($host =~ m/^([^:]+)(:[0-9]*)$/) {
      $host = $1;
      $port = "/" . $2;
    } else {
      $port = "";
    }

    # split host into pieces
    $hpart = "";
    while ($host =~ m|^(.*)\.([^.]+)$|) {
      $hpart .= ($2 . "/");
      $host = $1;
    }
    $hpart .= "$host";
    $hpart .= $port;
    $hpart .= "\@h";
  }

  $rv = $protocol . "/" . $hpart . "/" . $url;

  # escape characters that won't work in a UNIX pathname
  $rv =~ s|\.|\@pd|g;
  $rv =~ s|\"|\@dq|g; # "
  $rv =~ s|\'|\@sq|g;
  $rv =~ s|\$|\@dl|g;
  $rv =~ s|\~|\@tl|g;
  $rv =~ s|\-|\@hy|g;
  $rv =~ s|\ |\@sp|g;
  $rv =~ s|\&|\@am|g;
  $rv =~ s|\:|\@cl|g;
  $rv =~ s|\=|\@eq|g;
  $rv =~ s|\(|\@op|g;
  $rv =~ s|\)|\@cp|g;
  # other characters to eliminate:
  # ( )
  # These characters seem to be no problem in typing shell commands:
  # +

  # The '@' character was re-mapped at the beginning to avoid conflicts
  # with the conversion algorithm. Now we remap it again into its final form.
  $rv =~ s|\002|\@at|g;

  # divide path into smaller parts
  $rv2 = "";
  $rv .= "/";
  while ($rv =~ m|^([^/]+)/(.*)$|) {
    $unit = $1;
    $rv = $2;
    # unit might need to be divided
    while (length($unit) > 15) {
      if ($unit =~ m|^(.{12})(.?[^@])(.+)$|) {
        $rv2 .= "/" . $1 . $2 . "-";
        $unit = $3;
      } else {
        print "\n?? couldn't subdivide cache path $rv2 $unit\n";
        exit 0;
      }
    }
    $rv2 .= "/" . $unit;
  }
  $rv2 .= "/" . $rv;

  $rv = $rv2;

  # remove any extra /'s
  $rv =~ s|//|/|g;
  $rv =~ s|/+$||;
  $rv =~ s|^/||;

  # now remap the tag
  if ($tag ne "") {
    # first make sure we have a directory for the tag's location
    &cache_mkdir($rv);

    # check for duplicate, and generate a new tag ID
    open (TAGMAP, ".tagmap");
    $tid = 0; $lkg = 1;
    while (($l = <TAGMAP>) && $lkg) {
      chop $l;
      if ($tag eq $l) {
        $lkg = 0;
      }
      $tid += 1;
    }
    close TAGMAP;
    if ($lkg) {
      open (TAGMAP, ">> .tagmap");
      print TAGMAP "$tag\n";
      close TAGMAP;
      $tid += 1;
    }

    $tag = "\@qy" . $tid;

    # append tag to path
    $rv .= "/" . $tag;
  }

  # mark this cache path as being a file
  $rv .= "@";

  if ($full) {
    $rv = "$cachedir/$rv";
  }

  return $rv;
} # End of cache.path

# given a URL, returns the URL with the anchor or query (if any) removed
sub url_base
{
  my($url, $complete) = @_;

  if ($complete) {
    # it's okay to remove queries
    if ($url =~ m/([^?]+)\?(.*)$/) {
      $url = $1;
    }
  }

  # remove local anchors
  if ($url =~ m/([^#]+)#(.*)$/) {
    $url = $1;
  }

  return $url;
} # End of url.base

# given a URL, returns the beginning part (protocol and host) and without a
# trailing /
sub url_host
{
  my($url) = @_;

  if ($url =~ m|^(https?)://([^/]+)/|i) {
    $url = $1 . "://" . $2;
  }

  return $url;
}

sub apply_href
{
  my($url, $cmd, $ignore_null) = @_;
  my($cmd1);

  $cmd =~ s/^ +//;

  if (($cmd eq "") && $ignore_null) {
    $cmd = $url;
  } elsif ($cmd =~ m|^[a-z]+://|) {
    # all set
  } elsif ($cmd =~ m|^mailto:|) {
    # all set
  } elsif ($cmd =~ m|^news:|) {
    # all set
  } elsif ($cmd =~ m|^/|) {
    $cmd = &url_host($url) . $cmd;
  } elsif ($cmd =~ m/^\#/) {
    # remove existing anchor, if any, and append new one
    $cmd = &url_base($url, 1) . $cmd;
  } else {
    # start with base URL
    $cmd1 = $url;
    # remove existing leaf name if any
    if ($url =~ m|(.*/)([^/]*)$|) {
      $cmd1 = $1;
    }
    # and append new leaf name
    $cmd = $cmd1 . $cmd;
  }

  # Remove "/../" in path name. Path must have two components before the ".."
  # (A hostname and a directory). If not, we just remove the ..
ah1: ; 
 if ($cmd =~ m|/[^/]+/[^/]+/\.\./|) {
    $cmd1 = $cmd;
    $cmd1 =~ s|/[^/]+/\.\./|/|;
    if ($cmd1 ne $cmd) {
      $cmd = $cmd1;
      goto ah1;
    }
  } elsif ($cmd =~ m|/[^/]+/[^/]+/\.\.$|) {
    $cmd =~ s|/[^/]+/\.\.$|/|;
  } elsif ($cmd =~ m|/[^/]+/\.\./|) {
    $cmd =~ s|/\.\./|/|;
  }

  if ($cmd =~ m|/\./|) {
    $cmd =~ s|/\./|/|;
  }

  return $cmd;
} # End of apply.href

# This routine returns TRUE if the two URLs refer to the same data object.
# (anchors and query tags are ignored)
sub url_match
{
  my($url1, $url2) = @_;

  return (&url_base($url1, 1) eq &url_base($url2, 1));
} # End of url.match

# This routine returns TRUE if the given href points to a URL that has
# been visited. The base URL of the href must be given too. The third parameter
# is an index value into the global history; it will only count as a visit
# if the previous visit was more recent than this point in the global
# history; just pass 0 if you want everything to match.
sub url_visited
{
  my($url) = @_;
  my($v, $ur2, $rv);

  $ur2 = $url;
  $ur2 = &munge1($ur2);
  $ur2 = &url_base($ur2, 0);
  $v = $visited{$ur2};
  $rv = 0;
  if ($v > $vl_expire) {
    $rv = 1;
  }

  return $rv;
} # End of url.visited

sub pr2
{
  my($l) = @_;

  $l =~ s/\001/\n/g;
  $l =~ s/\002/$mk_y/g;
  $l =~ s/\003/$mk_b/g;
  $l =~ s/\004/$mk_i/g;
  $l =~ s/\005/$mk_n/g;

  print $l;
} # End of pr2

# This routine spools the output into OUTFILE and prints to the screen
# as well, unless some form of redirection is in effect.
#
sub proc16
{
  my($l_screen, $l_ascii, $l_rhtf) = @_;
  my($il, $do_x);

  print OUTFILE $l_screen;
  if ($of_more) {
    # nothing (child process does all printing and interaction)
  } elsif ($view_source) {
    # nothing (already been printed by pr2 in a previous filter stage)
  } else {
    if ($lof_going) {
      print $l_screen;
    }
    if ($l_screen =~ m/\n/) {
      $more_linecount++;
      if (($more_linecount > $screen_height) && ($defer_cmds eq "")) {
        print ("\n$magenta" . "More (A to spool all, X to skip):$g_normal ");
        $il = <>; chop $il;
        $il =~ tr/A-Z/a-z/;
        $do_x = 0;
        if ($il eq "x") {
          $do_x = 1;
        } elsif ($il eq "a") {
          $more_linecount = -9000000000;
        } elsif ($il eq "") {
          $more_linecount = 0;
        } else {
          # If it's unrecognized, treat it as another command, after
          # aborting load of current page.
          $do_x = 1;
          $defer_cmds .= (";" . $il);
        }

        if ($do_x) {
          $lof_going = 0;
          $more_linecount = -9000000000;
          if ($spoolreal) {
            print ("$g_normal" . "Please wait while skipping...");
          } else {
            print ("$g_normal" . "Skipping rest of file.");
          }
        }
      }
    }
  }

  print ASCIIFILE "$l_ascii";

  # Accumulate RHTF atoms into whole lines, to facilitate simplifying
  # span markup
  if ($l_rhtf eq "\n") {
    # Here we would substitute <*:...:> with *...*
    while ($rhtf_accum =~ m|^(.*)\<\*\:([^:]+)\:\>(.*)$|) {
      $rhtf_accum = $1 . "*" . $2 . "*" . $3;
    }
    print RHTFFILE "$rhtf_accum\n";
    $rhtf_accum = "";
  } else {
    $rhtf_accum .= $l_rhtf;
  }
} # end of proc.16

# Remove extra blank lines, then pass the rest to proc16
#
sub proc14
{
  my($l_screen, $l_ascii, $l_rhtf) = @_;

  if ($l_ascii eq "\n") {
    if ($p14_curblank) {
      # The previous line was blank
      if ($p14_lincnt < 0) {
        $p14_lincnt = 1;
      } else {
        # This is a second or subsequent newline
        $p14_lincnt++;
        # Discard this newline
        return;
      }
    } else {
      $p14_lincnt = -999999999;
    }

    # A newline gives the possibility of starting a new blank block
    $p14_curblank = 1;
  } elsif ($l_ascii =~ m/^ *$/) {
    # Absence of printable chars means that if we're in a blank block,
    # it remains valid
  } else {
    # Anything else ends any presently-active blank block
    $p14_curblank = 0;
  }

  &proc16($l_screen, $l_ascii, $l_rhtf);
} # End of proc.14

sub indent
{
  my($n) = @_;

  $bq_indent = (($bq_indent + $n) < 40) ? $bq_indent + $n : 40;
}

sub exdent
{
  my($n) = @_;

  $bq_indent = (($bq_indent - $n) >= 0) ? $bq_indent - $n : 0;
}

# This step counts the length of the formatted output for word wrapping,
# and then translates special control characters into escape sequences
# to create the boldface and colors, etc. Then it handles wrapping and
# passes the data on to proc14.
#
sub proc12
{
  my($l, $rhl) = @_;
  my($llen,$l2,$l3,$l4,$hr1,$hr2);

  # handle view source
  if ($view_source == 12) {
    &pr2("$mk_pr2$l");
    return;
  }

  # The two <hr> escapes can be done right away because they translate
  # directly into plain ASCII text.
  if ($l =~ m/[\021\022]/) {
    $hr1 = "-" x ($screen_width - $charpos);
    $hr2 = "=" x ($screen_width - $charpos);
    $l =~ s/\021/$hr1/;
    $l =~ s/\022/$hr2/;
  }

  $l =~ tr/\006/ /;

  # l2 is used to compute width for word wrapping (it has all controls
  # stripped out)
  # l3 is used to create text-only output (it has newlines but no other
  # controls).
  # l4 is for RHTF output
  $l4 = $l3 = $l2 = $l;
  $l3 =~ s/\001/\n/g;
  $l2 =~ s/[\001-\010\013-\014\016-\024]//g;
  $l3 =~ s/[\002-\010\013-\014\016-\024]//g;
  $llen = length($l2);

  $l =~ s/\001/\n/g;
  $l =~ s/\002/$yellow/g;
  $l =~ s/\003/$g_bold/g;
  $l =~ s/\004/$inverse/g;
  $l =~ s/\005/$g_normal/g;
  # 006 reserved for forced blank space
  $l =~ s/\007/$cyan/g;
  $l =~ s/\010/$g_visited_colour/g;
  # 011 is reserved for tab
  # 012 is reserved for newline
  $l =~ s/\013/$g_visited_bold/g;
  $l =~ s/\014/$bdcy/g;
  # 015 is reserved for CR
  $l =~ s/\016/$bdbl/g;
  $l =~ s/\017/$bdbl/g;
  $l =~ s/\020/$bdmag/g;
  $l =~ s/[\023\024]/*/g;

  if ($rhl ne "") {
    # We have special RHTF output (currently only used for the vector
    # component of an anchor tuple).
    $l4 = $rhl;
  } else {
    $l4 =~ s/\001/\n/g;
    $l4 =~ s/\002/#/g;
    $l4 =~ s/\003/#/g;
    $l4 =~ s/\004/#/g;
    $l4 =~ s/\005/#/g;
    $l4 =~ s/\023/<*:/g;
    $l4 =~ s/\024/:>/g;
    $l4 =~ s/[\002-\010\013-\014\016-\022]//g;
  }

  # short-circuit preformatted text
  if ($pre_f) {
    &proc14($pre_indent . $l, $pre_indent . $l3, $pre_indent . $l4);
    return;
  }

  $max = $screen_width;
  if ($bq_indent) {
    $max = $max - $bq_indent;
  }

  if ($l eq "\n") {
    &proc14($l, "\n", "\n");
    $charpos = 0;
  } else {
    if ($charpos + $llen > $screen_width) {
      &proc14("\n", "\n", "\n");
      $charpos = 0;
      if ($bq_indent) {
        &proc14(" " x $bq_indent, " " x $bq_indent, " " x $bq_indent);
        $charpos = $bq_indent;
      }
    }
    &proc14($l, $l3, $l4);
    $charpos += $llen;
  }
} # end of proc.12

# Splits line into words and passes along to proc12.
sub proc10
{
  my($l, $rhl) = @_;

  # handle view source
  if ($view_source == 10) {
    &pr2("$mk_pr2$l");
    return;
  }

  # If we have special RHTF output (currently only used for the vector
  # component of an anchor tuple), we bypass the word-wrapping. This
  # is intentional, because RHTF expects an anchors to be entirely on
  # one line.
  if ($rhl ne "") {
    &proc12($l, $rhl);
    return;
  }

  # Send out pieces that end in one or more spaces
  while (($spoolreal || $lof_going) && ($l =~ m|^([^ ]* +)(.*)$|)) {
    $l1 = $1;
    $l = $2;
    &proc12($l1);
  }
  # Send out the rest
  if ($spoolreal || $lof_going) {
    &proc12($l);
  }
} # End of proc.10

# proc09 handles newline commands (<p>, <br> and <tr>) including
# indentation and ignoring multiple newlines in a row. It also does
# horizontal bars (<hb>).
sub proc09
{
  my($l, $rhl) = @_;
  my($isbr);

  # handle view source
  if ($view_source == 9) {
    &pr2("$mk_pr2$l");
    return;
  }

  # If we have special RHTF output (currently only used for the vector
  # component of an anchor tuple), we bypass the processing. This
  # should be okay because <p>, etc. shouldn't occur inside the anchor
  # vector.
  if ($rhl ne "") {
    &proc10($l, $rhl);
    return;
  }

  # Operators that are handled even inside <pre> ... </pre> blocks

  $newline = "\001";

  # flag that this was a non-breaking line. We'll set $isbr if we print a
  # newline and suppress printing a newline next time.
  $isbr = 0;

  if ($l =~ m|<span[ >]|) {
    # other spans -- ignore
    return;
  } elsif ($l eq "</span>") {
    # ignore
    return;
  } elsif ( 0
   || ($l eq "<br>")
   || ($l =~ m|<div[ >]|)
   || ($l eq "<p>")
   || ($l =~ m|<tr[ >]|)
      # need to match both <tr> and </tr> because some web pages
      # omit their </tr>'s.
   || ($l =~ m|</tr[ >]|)
  ) {
    if ($didbr == 0) {
      &proc10($newline);
    }
    $isbr = 1;
    $didbr = $isbr;
    if ($l eq "<p>") {
      &proc10("  ");
    } elsif ($l =~ m|<div[ >]|){
      &proc10("\n");
    }
    return;
  }

  # short-circuit preformatted text
  if ($pre_f) {
    &proc10($l);
    $didbr = $isbr;
    return;
  }

  # get rid of excess blank space
  $l =~ s/\t/ /g;
  $l =~ s/ +/ /g;
  $l = "" if ($l eq " ");

  if ($kill_ls) {
    $l =~ s/^ *//;
  }

  # If line is empty, ignore it
  if ($l eq "") {
    return;
  }

  $kill_ls = 0;

  if ( 0
   || ($l =~ m|^<p |)
   || ($l =~ m|^<br |)
   || ($l =~ m|</head[ >]|)
   || ($l eq "<dl>")
  ) {
    if ($didbr == 0) {
      &proc10($newline);
    }
    $isbr = 1;
    $didbr = $isbr;
    return;
  } elsif ($l =~ m|<title[ >]|) {
    &proc10($zb[$in_anchor]);
    $titling = 1;
    $didbr = $isbr;
    return;
  } elsif ($l =~ m|</title[ >]|) {
    &proc10($zb[$in_anchor + $in_bold]);
    $titling = 0;
    $didbr = $isbr;
    return;
  } elsif ($l =~ m|<hr 1>|) {
    # Note: "<hr 1>" and "<hr 2>" are internal to MIRA and only exist between
    # proc.06 and proc.09. Actual "<hr" tags are filtered out by proc09 or
    # short-circuited as preformatted text.
    # Do a row of "-" to the end of the line.
    &proc10($ruletoend1);
  } elsif ($l =~ m|<hr 2>|) {
    # Do a row of "=" to the end of the line.
    &proc10($ruletoend2);
  } else {
    &proc10(" " x $bq_indent) if ($bq_indent && $didbr);
    &proc10($l);
    $page_title .= $l if ($titling);
    $didbr = $isbr;
    return;
  }

  $didbr = $isbr;
} # end of proc09

sub remap_chars
{
  my($l) = @_;
  my($c, $f1);

  # Note on accented characters: If we need an all-ASCII translation,
  # might want to use the syntax <'e> (for eacute)

  # UTF-8 decoding.
  #
  # 3-byte sequences need to be removed first, then 2-byte sequences, then
  # 8-bit characters and finally any 7-bit ASCII like control characters.

if(!($hdr_iso_8859_1)) {
  if ($l =~ m/\xc2/) {
    $l =~ s/\xc2\xa0/ /g;
  }

  if ($l =~ m/\xc3/) {
    $l =~ s/\xc3\x97/x/g;
  }

  if ($l =~ m/\xe2/) {
    $l =~ s/\xe2\x80\x89/ /g;
    $l =~ s/\xe2\x80\x90/-/g;
    $l =~ s/\xe2\x80\x91/-/g;
    $l =~ s/\xe2\x80\x92/-/g;
    $l =~ s/\xe2\x80\x93/-/g;
    $l =~ s/\xe2\x80\x94/--/g;
    $l =~ s/\xe2\x80\x95/--/g;
    $l =~ s/\xe2\x80\x98/'/g;
    $l =~ s/\xe2\x80\x99/'/g;
    $l =~ s/\xe2\x80\x9c/"/g;
    $l =~ s/\xe2\x80\x9d/"/g;
    $l =~ s/\xe2\x80\xa6/.../g;
    $l =~ s/\xe2\x88\x92/-/g;
  }

  if (($l =~ m/\xe1/) || ($l =~ m/\xe2/)) {
    print "\n\n${yellow}Unhandled 3-byte UTF-8 sequence$g_normal:\n   '";
    foreach $c (split(//, $l)) {
      if ((ord($c) >= 32) && (ord($c) < 127)) {
        print $c; $f1 = 0;
      } else {
        print " " unless ($f1);
        printf("${yel7}[0x%02x]$g_normal", ord($c));
        $f1 = 1;
      }
    }
    print "'\n";
    # %%% If this gets to be a pain, we can add an interactive prompt here to
    # disable further UTF-8 errors.
    # exit(1);
  }
}

  # print "foo1 $l\n" if ($l =~ m/1[\326-\327]2/);
  # print "foo2 $l\n" if ($l =~ m/1\3262/);
  # print "foo3 $l\n" if ($l =~ m/1\3272/);

  # This encoding of {x} appears in the factorial expansion examples on
  # http://www.contracosta.edu/math/powersOfTen.htm
  # %%% However, the raw HTML has the 2-byte UTF-8 sequence
  # \xc3\x97 = \303\227 so I
  # need to find out how that is getting turned into \xd7 = \327

  if ($hdr_iso_8859_1) {
    # These mappings appear in pages with "charset=iso-8859-1" in a meta
    # header, including pages that actually use Windows-1252.
    # See:
    #   en.wikipedia.org/wiki/ISO/IEC_8859-1
    #   en.wikipedia.org/wiki/Windows-1252
    #   www.w3schools.com/html/html_charset.asp
    $l =~ s|\x91|`|g;
    $l =~ s|\x92|'|g;
    $l =~ s|\x93|"|g;
    $l =~ s|\x94|"|g;
    $l =~ s|\xd7|x|g;
    $l =~ s|\xe2|^a|g;
  }

  if (0) {
    # This is some other character set, I don't know which
    $l =~ s|\325|'|g;
  }

  if ($l =~ m/&/) {
    $l =~ s/&#9;/ /g; # this is really a tab character, but I don't care

    # Normal ASCII characters, even printable ones, often appear encoded.
    # Care to guess why someone might encode the letters e,s,x? (-:
    $l =~ s/&#34;/\"/g; # "
    $l =~ s/&#36;/\$/g;
    $l =~ s/&#38;/\&/g;
    $l =~ s/&#39;/\'/g;
    $l =~ s/&#40;/(/g;
    $l =~ s/&#41;/)/g;
    $l =~ s/&#59;/;/g;
    $l =~ s/&#91;/[/g;
    $l =~ s/&#93;/]/g;
    $l =~ s/&#101;/e/g;
    $l =~ s/&#115;/s/g;
    $l =~ s/&#120;/x/g;
    $l =~ s/&#124;/|/g;

    # 8-bit extended ASCII: These seem to be mostly Windows-1252
    $l =~ s/&#133;/.../g;
    $l =~ s/&#0133;/.../g;
    $l =~ s/&#134;/  /g;   # This is just a guess
    $l =~ s/&#145;/`/g;
    $l =~ s/&#146;/'/g;
    $l =~ s/&#0146;/'/g;
    $l =~ s/&#147;/"/g;
    $l =~ s/&#0147;/"/g;
    $l =~ s/&#148;/"/g;
    $l =~ s/&#0148;/"/g;
    $l =~ s/&#149;/*/g;
    $l =~ s/&#0149;/*/g;
    $l =~ s/&#150;/ /g;    # This is just a guess
    $l =~ s/&#151;/--/g;
    $l =~ s/&#0151;/--/g;
    $l =~ s/&#151/--/g;
    $l =~ s/&#153;/^{TM}/g;
    $l =~ s/&#153/^{TM}/g;
    $l =~ s/&#0?160;?/ /g;
    $l =~ s/&#162;/\242/g;
    $l =~ s/&#0162;/\242/g;
    $l =~ s/&#169;/\251/g;
    $l =~ s/&#169/\251/g;
    $l =~ s/&#173;/-/g;
    $l =~ s/&#174;/\256/g;
    $l =~ s/&#176;/\260/g;
    $l =~ s/&#183;/-/g;
    $l =~ s/&#185;/\271/g; # ^{1}
    $l =~ s/&#0189;/\275/g; # 1/2
    $l =~ s/&#198;/AE/g;
    $l =~ s/&#215;/x/g;
    $l =~ s/&#224;/\340/g;
    $l =~ s/&#226;/\342/g;
    $l =~ s/&#230;/ae/g;
    $l =~ s/&#233;/\351/g;
    $l =~ s/&#241;/\361/g;
    $l =~ s/&#246;/\366/g;
    $l =~ s|&#247;|/|g;
    $l =~ s/&#252;/\374/g;

    # Unicode
    $l =~ s/&#8211;/--/g;
    $l =~ s/&#8212;/--/g;
    $l =~ s/&#8216;/'/g;
    $l =~ s/&#8217;/'/g;
    $l =~ s/&#8220;/"/g;
    $l =~ s/&#8221;/"/g;
    $l =~ s/&#8230;/.../g;

    $l =~ s/&Uuml;/:U/g;
    $l =~ s/&uuml;/:u/g;
    $l =~ s/&ugrave;/`u/g;
    $l =~ s/&ucirc;/"u/g;
    $l =~ s/&uacute/'u/g;
    $l =~ s/&THORN;/TH/g;
    $l =~ s/&thorn;/th/g;
    $l =~ s/&szlig;/ss/g; # German double-s
    $l =~ s/&sup2;/^{2}/g;
    $l =~ s/&sup1;/^{1}/g;
    $l =~ s/&shy;/-/g;
    $l =~ s/&rsquo;/'/g; # '
    $l =~ s/&reg;/(r)/g;
    $l =~ s/&rdquo;/"/g; # "
    $l =~ s/&rarr;/->/g;
    $l =~ s/&hellip;/.../g;
    $l =~ s/&quot;/"/g;
    $l =~ s/&quot/"/g;
    $l =~ s/&pound;/-L/g; # Pound sign
    $l =~ s/&Ouml;/:O/g;
    $l =~ s/&ouml;/:o/g;
    $l =~ s/&otilde;/~o/g;
    $l =~ s/&oslash;/\/o/g;
    $l =~ s/&ograve;/`o/g;
    $l =~ s/&ocirc;/"o/g;
    $l =~ s/&oacute;/'o/g;
    $l =~ s/&ntilde;/~n/g;
    $l =~ s/&nbsp;/\006/g;
    $l =~ s/&nbsp/\006/g;
    $l =~ s/&mdash;/ -- /g;
    $l =~ s/&middot;/*/g;
    $l =~ s/&lt;/</g;
    $l =~ s/&lt/</g;
    $l =~ s/&lsquo;/'/g; # '
    $l =~ s/&ldquo;/"/g; # "
    $l =~ s/&Iuml;/:I/g;
    $l =~ s/&iuml;/:i/g;
    $l =~ s/&Igrave;/`I/g;
    $l =~ s/&igrave;/`i/g;
    $l =~ s/&icirc;/"i/g;
    $l =~ s/&iacute;/'i/g;
    $l =~ s/&gt;/>/g;
    $l =~ s/&gt/>/g;
    $l =~ s/&euml;/:e/g;
    $l =~ s/&eth;/-d/g; # Icelandic "eth"
    $l =~ s/&egrave;/`e/g;
    $l =~ s/&ecirc;/"e/g;
    $l =~ s/&Eacute;/'E/g;
    $l =~ s/&eacute;/'e/g;
    $l =~ s/&deg;/"/g;
    $l =~ s/&copy;/(c)/g;
    $l =~ s/&ccedil;/,c/g;
    $l =~ s/&auml;/:a/g;
    $l =~ s/&atilde;/~a/g;
    $l =~ s/&aring;/"a/g;
    $l =~ s/&amp;/\&/g;
    $l =~ s/&agrave;/`a/g;
    $l =~ s/&aelig;/ae/g;
    $l =~ s/&acirc;/^a/g;
    $l =~ s/&aacute;/'a/g;
  }

  return $l;
} # End of remap.chars

# remap special characters like "&copy;" into ASCII equivalents.
sub proc08
{
  local($l, $rhl) = @_;

  # handle view source
  if ($view_source == 8) {
    &pr2("$mk_pr2$l");
    return;
  }

  # If we have special RHTF output (currently only used for anchors)
  # we bypass the remapping.
  if ($rhl ne "") {
    &proc09($l, $rhl);
    return;
  }

  # short-circuit preformatted text
#  if ($pre_f) {
#    &proc09($l, $l);
#    return;
#  }

  $l = &remap_chars($l);

  &proc09($l);
} # End of proc.08

# Used for <img>
sub add_image
{
  local($load_url, $i_href) = @_;

  if ($rev_img{"$load_url $i_href"}) {
    $this_img_num = $rev_img{"$load_url $i_href"};
  } else {
    $img_num += 1;
    $this_img_num = $img_num;
    $img_src[$img_num] = $i_href;
    $img_base[$img_num] = $load_url;
    $rev_img{"$load_url $i_href"} = $img_num;
  }
} # End of add.image

# Used for <embed>
sub add_embed
{
  local($load_url, $i_href) = @_;

  if ($rev_embed{"$load_url $i_href"}) {
    $this_embed_num = $rev_embed{"$load_url $i_href"};
  } else {
    $embed_num += 1;
    $this_embed_num = $embed_num;
    $embed_src[$embed_num] = $i_href;
    $embed_base[$embed_num] = $load_url;
    $rev_embed{"$load_url $i_href"} = $embed_num;
  }
} # End of add.embed

# Used for <select> and <option> to pick up option values given inline
# rather than with value= fields
sub check_foption
{
  if ($f_ovkey ne "") {
    $f_value =~ s/[\001-\037\200-\377]//g;
    $opt_value{$f_ovkey} = $f_value;
    $f_ovkey = "";
    if ($f_selected) {
      $input_value[$inp_num] = $f_value;
    }
  }
} # End of check.foption

# Checks to see if anchor is already defined, to avoid defining multiple
# redundant anchor numbers
sub add_anchor1
{
  local($load_url, $a_hrf) = @_;

  if ($a_hrf =~ m|^javascript:openWindow\('(.*)'\);$|) {
    $a_hrf = $1;
  } elsif ($a_hrf =~ m|^javascript:newWin\('(.*)'|) {
    $a_hrf = $1;
  } elsif ($a_hrf =~ m|^javascript:openDocumentWindow\('(.*)'|) {
    $a_hrf = $1;
  } elsif ($a_hrf =~ m|^javascript:window.mNW\('(.*)'|) {
    $a_hrf = "CloserLook.asp?" . $1;
  }

  if ($rev_anc{"$load_url $a_hrf"}) {
    $this_ancnum = $rev_anc{"$load_url $a_hrf"};
  } else {
    $anc_num += 1;
    $this_ancnum = $anc_num;
    $anchors[$anc_num] = $a_hrf;
    $anc_base[$anc_num] = $lnc_url;
    $rev_anc{"$load_url $a_hrf"} = $anc_num;
  }

  $last_href = $a_hrf;

  return $this_ancnum;
} # ENd of add.anchor1

# here's where we determine whether to use LINK or VLINK color.
# if visited or if it's a link to this page, it's colored green,
# else blue or light blue depending on whether the link is on the
# same host.
sub acolor
{
  local($chref, $load_url) = @_;
  local($rv);

  if (&url_visited($chref)) {
    $rv = 6;
  } elsif (&url_base($chref, 0) eq &url_base($load_url, 0)) {
    $rv = 6;
  } else {
    if (&url_host($chref) eq &url_host($load_url)) {
      $rv = 4;
    } else {
      $rv = 2;
    }
  }

  return $rv;
}

# Used for <a>, <area> and <meta REFRESH>. Always calls add_anchor1 once on
# the literal href text. If the href is a query, it also tries to glean a URL
# from within query fields.
# 
sub add_anchor2
{
  local($load_url, $a_hrf) = @_;
  local($href2, $ia, $l, $glean);

  $href2 = $a_hrf;
  if ($href2 =~ m|%|) {
    $href2 =~ s|%7e|~|gi;
    $href2 =~ s|%5f|_|gi;
    $href2 =~ s|%3a|:|gi;
    $href2 =~ s|%2f|/|gi;
    $href2 =~ s|%2e|.|gi;
    $href2 =~ s|%2d|-|gi;
  }

  $glean = 0;
  if (($href2 . "&") =~ m|\?.+\=(http://[^&]+)\&|i) {
    $href2 = $1;
    $glean = 1;
  } elsif ($href2 =~ m|/(ftp://[^&]+)\?|) {
    $href2 = $1;
    $glean = 1;
  }

  if ($glean) {
    $l = "$zm\{";
    $ia = &acolor($href2, $load_url);
    $l .= $zb[$ia + 1];
    $l .= &add_anchor1($load_url, $href2);
    $l .= "$zm}$zn";
    &proc08($l);
  }

  &add_anchor1($load_url, $a_hrf);
}

sub close_anchor
{
  local($l);

  $l = $zb[$in_anchor + $in_bold];
  $l .= "|";
  $l .= "$zi$this_ancnum$zn";
  $l .= $zb[$in_bold];
  $l .= "]";
  $in_anchor = 0;

  return $l;
}

sub clip20
{
  local($l) = @_;

  if ($l =~ m|(.{20})$|) {
    $l = "..." . $1;
  } else {
    # nothing
  }
  return $l;
}

# this routine checks to see if a frame URL is unique and adds it to the
# frame list.
sub add_frame
{
  local($cur_url, $href) = @_;
  local($unique, $i);

  $href = &apply_href($cur_url, $href, 1);
  $unique = 1;
  for($i=1; $i<=$num_frames; $i++) {
    if ($href eq $frame_url[$i]) {
      $unique = 0;
      $i = $num_frames+1;
    }
  }

  if ($unique) {
    $num_frames += 1;
    $frame_url[$num_frames] = $href;
  }
}

# convert links, images, EMBED (e.g. QuickTime movies), forms inputs, etc.
sub proc07
{
  local($l) = @_;
  local($v, $hr2, $a_hrf, $a_nme);

  if ($ignore) {
    return;
  }

  # handle view source
  if ($view_source == 7) {
    &pr2("$mk_pr2$l");
    return;
  }

  # Operators that are handled even inside <pre> ... </pre> blocks

  # handle anchors "<a " and </a>
  if ($l =~ m/^<a .*href *= *\"([^"]+)\"/i) { # "
    $a_hrf = $1;
    goto anc1;
  } elsif ($l =~ m/^<a .*href *= *\'([^']+)\'/i) {  # '
    $a_hrf = $1;
    goto anc1;
  } elsif ($l =~ m/^<a .*href *= *([^ >]+)[ >]/i) {
    $a_hrf = $1;
anc1: ;
    if ($a_hrf =~ m/^#/) {
      $a_hrf =~ s/^#/#l_/;
    }
    &add_anchor2($load_url, $a_hrf);
    $hr2 = &munge1(&apply_href($load_url, $a_hrf, 1));
    $in_anchor = &acolor($hr2, $load_url);
    $l = " [" . $zb[$in_anchor + $in_bold];
    &proc08($l);
    $kill_ls = 1;
    return;

  } elsif ($l =~ m/^<a .*name *= *\"([^"]+)\"/i) { # "
    $a_nme = $1;
    goto anc2;
  } elsif ($l =~ m/^<a .*name *= *\'([^']+)\'/i) { # '
    $a_nme = $1;
    goto anc2;
  } elsif ($l =~ m/^<a .*name *= *([^ >]+)[ >]/i) {
    $a_nme = $1;
anc2: ;
    $l = "l_" . $a_nme . "::";
    # Make sure we have just done a break, then do another
    &proc08("<br>"); $didbr = 0; &proc08("<br>");
    &proc08($l);
    &proc08("<br>");
    return;

  } elsif ($l =~ m/^<a /) {
    $l = "";
  } elsif ($l =~ m|</a[ >]|) {
    if ($in_anchor) {
      # Here we use a short-circuit signal to bypass all the formatting
      # between ourselves and proc12.
      $rhl = '|' . $anchors[$anc_num] . ']';
      &proc08(&close_anchor(), $rhl);
      return;
    } else {
      $l = "";
    }
    $in_anchor = 0;
  } else {
    $anc_text[$this_ancnum] .= $l if ($in_anchor);
  }

  # handle images
  if (($l =~ m/^<img /) || ($l =~ m/^<image /)) {
    # extract alt
    $img_alt = "";
    if ($l =~ m/.*alt *= *\"([^"]+)\"/i) { # "
      $img_alt = $1;
    }

    # extract source (which is a relative URL like an href)
    $source = "";
    if ($l =~ m/.*src *= *\"([^"]+)\"/i) { # "
      $source = $1;
    } elsif ($l =~ m|.*src *= *([^ >]+)[ >]|i) {
      $source = $1;
    }

    &add_image($lnc_url, $source);
    $l = "\{$zi" . "im $this_img_num$zn";
    $l .= $zb[$in_anchor + $in_bold];
    $l .= "$img_alt\}";
  }

  # look for QT movies
  if ($l =~ m/^<embed /) {
    # extract source (which is a relative URL like an href)
    $embed_src = "";
    if ($l =~ m/.*src *= *\"([^"]+)\"/i) { # "
      $embed_src = $1;
    } elsif ($l =~ m|.*src *= *([^ >]+)[ >]|i) {
      $embed_src = $1;
    }

    &add_embed($lnc_url, $embed_src);

    $l = "\{$zi" . "dl em $this_embed_num $zn";
    $l .= $zb[$in_anchor + $in_bold];
    $l .= "$embed_src\}";
  }

  # handle select and option
  if ($l =~ m/^<select /) {
    &check_foption;

    $inp_num += 1;
    $opt_num = 0;

    # extract name
    $name = "";
    if ($l =~ m/.*name *= *([\.A-Za-z]+)[^\.A-Za-z]/i) {
      $name = $1;
    } elsif ($l =~ m/name *= *\"([^"]+)\"/i) { # "
      $name = $1;
    }

    # Flag this input as being a <select>
    $input_type[$inp_num] = "select";
    $input_form[$inp_num] = $form_stack[0];
    $input_name[$inp_num] = $name;

    $l = "<br>";
  } elsif ($l =~ m/^<option .*value *= *\"([^"]*)\"/i) { # "
    &check_foption;

    $value = $1;

    $opt_num += 1;
    $opt_value{"$inp_num.$opt_num"} = $value;

    if ($retain_form) {
      # don't change value
    } elsif ($l =~ m/selected/i) {
      $input_value[$inp_num] = $value;
    }

    $l = " " . $zi . "i$inp_num.o$opt_num" . $zb[$in_anchor + $in_bold] . ":" . $zy;
  } elsif ($l =~ m/^<option .*value *= *([\.a-zA-Z0-9]+)/i) {
    &check_foption;

    $value = $1;

    $opt_num += 1;
    $opt_value{"$inp_num.$opt_num"} = $value;

    if ($retain_form) {
      # don't change value
    } elsif ($l =~ m/selected/i) {
      $input_value[$inp_num] = $value;
    }

    $l = " " . $zi . "i$inp_num.o$opt_num" . $zb[$in_anchor + $in_bold] . ":" . $zy;
  } elsif ($l =~ m|^<option[> ]|) {
    &check_foption;

    $opt_num += 1;
    $f_ovkey = "$inp_num.$opt_num";
    $f_value = "";
    $f_selected = ($l =~ m/selected/i);
    $opt_value{$f_ovkey} = $f_value;
    $l = " " . $zi . "i$inp_num.o$opt_num" . $zb[$in_anchor + $in_bold] . ":" . $zy;
  } elsif ($l =~ m|</select>|) {
    &check_foption;

    # nothing
    &proc08($zb[$in_anchor + $in_bold]);
    $l = "<br>";
  } else {
    if (length($f_value) < 200) {
      $f_value .= $l;
    }
  }

  # handle forms and inputs
  if ($l =~ m|^<form |) {
    $form_num += 1;
    unshift(@form_stack, $form_num);

    # extract method
    $method = "get";
    if ($l =~ m/.*method *= *([A-Za-z]+)[^a-zA-z]/i) {
      $method = $1;
      $method =~ tr/A-Z/a-z/;
    }

    # extract action (which is a relative URL like an href)
    $action = "";
    if ($l =~ m/.*action *= *\"([^"]+)\"/i) { # "
      $action = $1;
    }

    $form_action[$form_stack[0]] = $action;
    $form_method[$form_stack[0]] = $method;
    $form_base[$form_stack[0]] = $load_url;
    $l = " $zy" . "Form:$zb[$in_anchor + $in_bold]";
  } elsif ($l eq "</form>") {
    shift @form_stack;
    $l = "";
  } elsif (($l =~ m/^<input /) || ($l =~ m/^<textarea /)) {
    $inp_num += 1;

    # extract size
    $size = 0;
    if ($l =~ m/.*size *= *([0-9]+)[^0-9]/i) {
      $size = $1;
    }

    # extract type
    $type = "text";
    if ($l =~ m/.*type *= *([\.A-Za-z]+)[^\.A-Za-z]/i) {
      $type = $1;
      $type =~ tr/A-Z/a-z/;
    } elsif ($l =~ m/.*type *= *\"([^"]+)\"/i) { # "
      $type = $1;
      $type =~ tr/A-Z/a-z/;
    }

    # extract name
    $name = "";
    if ($l =~ m/.*name *= *\"([^"]+)\"/i) { # "
      $name = $1;
    } elsif ($l =~ m/.*name *= *([\.A-Za-z0-9]+)[^\.A-Za-z0-9]/i) {
      $name = $1;
    }

    # extract value
    $value = "";
    if ($l =~ m/.*value *= *\"([^"]+)\"/i) { # "
      $value = $1;
    } elsif ($l =~ m/.*value *= *([\.A-Za-z0-9]+)[^\.A-Za-z0-9]/i) {
      $value = $1;
    }

    # extract alt
    $alt = "";
    if ($l =~ m/.*alt *= *\"([^"]+)\"/i) { # "
      $alt = $1;
    }

    # extract checked
    $checked = " ";
    if ($l =~ m/checked/i) {
      $checked = "*";
    }

    if ($retain_form) {
      # don't change value, etc.
      $value = $input_value[$inp_num];
      $checked = $input_checked[$inp_num];
    } else {
      $input_value[$inp_num] = $value;
      $input_checked[$inp_num] = $checked;
    }
    $input_form[$inp_num] = $form_stack[0];
    $input_type[$inp_num] = $type;
    $input_name[$inp_num] = $name;

    if ($type eq "hidden") {
      $l = "";
    } elsif ($type eq "image") {
      $l = " $zi" . "i$inp_num".$zb[$in_anchor + $in_bold].":$zy$alt".$zb[$in_anchor + $in_bold];
    } elsif ($type eq "radio") {
      $l = " $zi" . "i$inp_num".$zb[$in_anchor + $in_bold]."($checked)";
    } elsif ($type eq "checkbox") {
      $l = " $zi" . "i$inp_num".$zb[$in_anchor + $in_bold] . "[" . $checked . "]";
    } else {
      $l = " $zi" . "i$inp_num".$zb[$in_anchor + $in_bold].":$zy" . "[" . $value;
      if ($size) {
        $l .= ("_" x ($size - length($value)));
      }
      $l .= "]" . $zb[$in_anchor + $in_bold];
    }
  }

  # short-circuit preformatted text
  if ($pre_f) {
    &proc08($l);
    return;
  }

  # handle areas
  if ($l =~ m/^<area .*href *= *\"([^"]+)\"/i) { # "
    $a_hrf = $1;

    &add_anchor2($load_url, $a_hrf);
    $label = &clip20($last_href);

    $hr2 = &munge1(&apply_href($load_url, $a_hrf, 1));
    $in_anchor = &acolor($hr2, $load_url);

    $l = " [$zb[$in_anchor]";
    $l .= "AREA$q$label$q" . $zb[$in_anchor + $in_bold] . "|$zi$this_ancnum";
    $l .= $zb[$in_bold]."]";
    $in_anchor = 0;
  } elsif ($l =~ m|<area |) {
    $l = "";
  } elsif ($l =~ m|</area>|) {
    $l = "";
  }

  # handle "meta" tags
  if ($l =~ m|^<meta *http-equiv="refresh|i) {
    if ($l =~ m|[0-9]+\ *; *URL\=(.+)\"|i) { # "
      $a_hrf = $1;
      &add_anchor2($load_url, $a_hrf);
      $l = "\n[$zb[$in_anchor]" . "TIMED REFRESH to $q" . &clip20($a_hrf) . $q
           . $zb[$in_anchor + $in_bold] . "|$zi$this_ancnum"
           . $zb[$in_anchor + $in_bold] . "]";
    }
  } elsif ($l =~ m|^<meta *name="description" content="(.*)"|i) {
    $l = $1;
    &proc08("<br>");
  } elsif ($l =~ m|^<meta *name="keywords" content="(.*)"|i) {
    $l = "Keywords: " . $1;
    &proc08("<br>");
  } elsif ($l =~ m|^<meta *name="author" content="(.*)"|i) {
    # ignored
    $l = "";
  } elsif ($l =~ m|^<meta *name="generator" content="(.*)"|i) {
    # ignored
    $l = "";
  } elsif ($l =~ m|^<meta *name="(.+)" content="(.*)"|i) {
    # all other content is printed out
    $l = $1 . ": " . $2 . " ";
  } elsif ($l =~ m|^<meta *http-equiv.*charset.*iso-8859-1|i) {
    # (Note: pages using Windows-1252 are often mistagged as ISO 8859-1)
    $hdr_iso_8859_1 = 1; $l = "";
  } elsif ($l =~ m|^<meta *http-equiv.*charset.+windows-1252|i) {
    # Also called "ANSI"
    $hdr_iso_8859_1 = 1; $l = "";
  } elsif ($l =~ m|^<meta *http-equiv.*charset.+utf-8|i) {
    # Definitely treat as UTF-8
    $hdr_iso_8859_1 = 0;
    $l = "";
  } elsif ($l =~ m|^<meta *http-equiv.*charset"|i) {
    # %%% In future we want to recognise other charsets
    $hdr_iso_8859_1 = 0;
    $l = "";
  } elsif ($l =~ m|^<meta *http-equiv="|i) {
    # ignored
    $l = "";
  } elsif ($l =~ m|<meta |) {
    $l = "";
  }

  # handle frames
  if ($l =~ m/^<frame .*src *= *\"([^"]+)\"/i) { # "
    &add_frame($cur_url, $1);
    $l = " {frame $num_frames}";
  } elsif ($l =~ m/^<frame .*src *= *([^ ]+)[ >]/i) {
    &add_frame($cur_url, $1);
    $l = " {frame $num_frames}";
  }

  &proc08($l);
} # end of proc07

# Filter out newlines (unless preformatted)
sub proc06
{
  local($l) = @_;

  # handle view source
  if ($view_source == 6) {
    &pr2("$mk_pr2$l");
    return;
  }

  if ($pre_f) {
    # leave it alone
  } else {
    $l =~ s/[\001\011]+/ /g;
  }

  &proc07($l) if ($l ne "");
} # end of proc06

# Reset flags that might be set by meta tags
sub reset_page_flags
{
  $hdr_iso_8859_1 = 0;
}

# handle simple atoms, like "_" for "<u>"
sub proc05
{
  local($l) = @_;

  # handle view source
  if ($view_source == 5) {
    &pr2("$mk_pr2$l");
    return;
  }

  # Operators that are handled even inside <pre> ... </pre> blocks
  if ( 0
    || ($l =~ m/^<article /)
    || ($l eq "</article>")

    || ($l =~ m/^<aside /)
    || ($l eq "</aside>")

    || ($l =~ m/^<footer /)
    || ($l eq "</footer>")

    || ($l =~ m/^<header /)
    || ($l eq "</header>")

    || ($l =~ m/^<main /)
    || ($l eq "</main>")

    || ($l =~ m/^<nav /)
    || ($l eq "</nav>")

    || ($l eq "<tbody>")
    || ($l eq "</tbody>")

    || ($l eq "<tt>")
    || ($l eq "</tt>")

    || ($l eq "<code>")
    || ($l =~ m/^<font /)

    || ($l eq "</code>")
    || ($l =~ m|</font[ >]|)
    || ($l =~ m|</table[ >]|)
  ) {
    # nothing
    return;
  } elsif ($l =~ m|<b[ >]|) {
    $in_bold = 1;
    &proc06($zb[$in_anchor + $in_bold]);
    return;
  } elsif ($l =~ m|<base HREF=\"([^\"]+)\">|) { # "
    if ($enable_redirects) {
      $lnc_url = $1;
      $this_url = $lnc_url if ($this_url eq "");
    }
    return;
  } elsif ($l =~ m|</b[ >]|) {
    $in_bold = 0;
    &proc06($zb[$in_anchor + $in_bold]);
    return;
  } elsif ($l =~ m|^<!doctype html|i) {
    &reset_page_flags();
    $pre_f = 0;
    return;
  } elsif ($l =~ m|<html[ >]|) {
    &reset_page_flags();
    $pre_f = 0;
    return;
  } elsif ($l =~ m|<head[ >]|) {
    &reset_page_flags();
    $pre_f = 0;
    return;
  } elsif ($l =~ m|<i[ >]|) {
    &proc06("\023");
    $kill_ls = 1;
    return;
  } elsif ($l =~ m|</i[ >]|) {
    &proc06("\024");
	return;
  } elsif ($l =~ m|<var[ >]|) {
    &proc06("\023");
    $kill_ls = 1;
    return;
  } elsif ($l =~ m|</var[ >]|) {
    &proc06("\024");
	return;
  } elsif ($l =~ m|<listing>|) {
    $pre_indent = "    ";
    &proc06("<br>");
    $pre_f = 1;
    return;
  } elsif ($l =~ m|</listing>|) {
    $pre_indent = "";
    $pre_f = 0;
    &proc06("<br>");
    return;
  } elsif ($l =~ m|<pre[ >]|) {
    $pre_f = 1;
    &proc06("\001");
    return;
  } elsif ($l =~ m|</pre>|) {
    &proc06("\001");
    $pre_f = 0;
    return;
  } elsif ($l eq "<strong>") {
    $in_bold = 1;
    &proc06($zb[$in_anchor + $in_bold]);
    return;
  } elsif ($l =~ m|</strong[ >]|) {
    $in_bold = 0;
    &proc06($zb[$in_anchor + $in_bold]);
    return;
  } elsif ($l eq "<sub>") {
    &proc06("v{");
    $kill_ls = 1;
    return;
  } elsif ($l eq "</sub>") {
    &proc06("}");
    return;
  } elsif ($l =~ m|<sup[ >]|) {
    &proc06("^{");
    $kill_ls = 1;
    return;
  } elsif ($l =~ m|</sup[ >]|) {
    &proc06("}");
    return;
  } elsif ($l eq "<table>") {
    &proc06("<br>");
    return;
  } elsif ($l =~ m/^<table /) {
    &proc06("<br>");
    return;
  } elsif ($l =~ m|<td[ >]|) {
    # The following would make indentation (extra blank spaces) appear
    # inside each table cell. I'm not sure why I ever wanted to do
    # that. -20000202
    # $didbr = 1;
    return;
  } elsif ($l =~ m|<th[ >]|) {
    # Treat <th> the same as <td>
    return;
  } elsif (($l =~ m|</td[ >]|) || ($l =~ m|</th[ >]|)) {
    if ($in_anchor) {
      &proc06(&close_anchor());
    }

    &proc06(" | ");
    return;
  } elsif ($l eq "<u>") {
    &proc06("_");
    return;
  } elsif ($l =~ m|<span.*style.*text-decoration.*underline.*>|) {
    &proc06("_");
    $in_span_ul = 1;
    return;
  } elsif ($l eq "</u>") {
    &proc06("_");
    return;
  } elsif ($in_span_ul && ($l eq "</span>")) {
    &proc06("_");
    $in_span_ul = 0;
    return;
  } elsif ($l =~ m|<!--.*-->|) {
    $l="";
    $ignore = 0;
  } elsif (($l =~ m|^<script |)
        || ($l eq "<script>")
        || ($l =~ m|^<style |)
        || ($l eq "<style>")
        || ($l =~ m|^<applet |)
        || ($l =~ m|<!--|)
   #    || ($l =~ m|^<noframe|)   Deliberate: I want to see the <noframe> text
        ) {
    $ignore = 1;
  } elsif (($l =~ m|^</script|)
        || ($l =~ m|^</style|)
        || ($l =~ m|^</applet|)
        || ($l =~ m|-->|)
   #    || ($l =~ m|^</noframe|)  Deliberate: I want to see the <noframe> text
        ) {
    $l = "";
    $ignore = 0;
  } elsif ($l =~ m|^</?noframe|) {
   # nothing
   return;
  } elsif ($l =~ m|<!|) {
    # one-line comment: do nothing
  }

  # short-circuit preformatted text
  if ($pre_f) {
    &proc06($l);
    return;
  }

  if ( 0
    || ($l eq "<address>")
    || ($l eq "<big>")
    || ($l eq "<body>")
    || ($l eq "<caption>")
    || ($l eq "<center>")
    || ($l eq "<cite>")
    || ($l eq "<kbd>")
    || ($l eq "<nobr>")
    || ($l eq "<noembed>")
    || ($l eq "<noscript>")
    || ($l eq "<samp>")
    || ($l eq "<small>")
    || ($l eq "<th>")

    || ($l eq "</address>")
    || ($l eq "</big>")
    || ($l eq "</body>")
    || ($l eq "</br>")
    || ($l eq "<!br>")
    || ($l eq "</caption>")
    || ($l eq "</center>")
    || ($l eq "</cite>")
    || ($l eq "</dd>")
    || ($l eq "</dt>")
    || ($l eq "</frame>")
    || ($l eq "</frameset>")
    || ($l eq "</html>")
    || ($l eq "</iframe>")
    || ($l eq "</kbd>")
    || ($l eq "</map>")
    || ($l eq "</marquee>")
    || ($l eq "</mvassign>")
    || ($l eq "</nobr>")
    || ($l eq "</noembed>")
    || ($l eq "</noscript>")
    || ($l eq "</object>")
    || ($l eq "</option>")
    || ($l eq "</samp>")
    || ($l eq "</small>")
    || ($l eq "</span>")
    || ($l eq "</style>")
    || ($l eq "</textarea>")

    || ($l =~ m/^<align/)
    || ($l =~ m/^<basefont /)
    || ($l =~ m/^<bgsound /)
    || ($l =~ m/^<big /)
    || ($l =~ m/^<body /)
    || ($l =~ m/^<frameset /)
    || ($l =~ m/^<iframe /)
    || ($l =~ m/^<link /)
    || ($l =~ m/^<map /)
    || ($l =~ m/^<marquee /)
    || ($l =~ m/^<o\:/)
    || ($l =~ m/^<object /)
    || ($l =~ m/^<param /)
    || ($l =~ m/^<small /)
    || ($l =~ m/^<spacer /)
    || ($l =~ m/^<span /)
    || ($l =~ m|^<style[ >]|)
    || ($l =~ m|^<th[ >]|)

    || ($l =~ m|^<noindex[ >]|)

    || ($l =~ m|^</div[ >]|)
    || ($l =~ m|^</noindex[ >]|)
    || ($l =~ m|^</o\:|)
    || ($l =~ m|^</p[ >]|)
    || ($l =~ m|^</th[ >]|)
  ) {
    # nothing
  } elsif ($l =~ m|</?h[0-6][> ]|) {
    &proc06("<br>");
  } elsif ($l eq "<blink>") {
    $in_bold = 1;
    &proc06($zb[$in_anchor + $in_bold]);
    return;
  } elsif ($l eq "</blink>") {
    $in_bold = 0;
    &proc06($zb[$in_anchor + $in_bold]);
    return;
  } elsif ($l =~ m|<blockquote[ >]|) {
    &proc06("<br>");

    &indent(4);
  } elsif ($l =~ m|</blockquote>|) {
    &exdent(4);
    &proc06("<br>");
  } elsif ($l =~ m|^<ol[ >]|) {
    &proc06("<br>");
    $in_ol = 1;
    $li_num = 0;
  } elsif ($l =~ m|</ol>|) {
    $in_ol = 0;
    &proc06("<br>");
  } elsif ($l eq "<dd>") {  # http://www.rickross.com/reference/ICSD23.html  -- appears to be just a <p> plus nonbreaking space
    &proc06("<br>");
    &proc06("\006\006");
    &indent(4);
  } elsif ($l eq "</dd>") {
    &exdent(4);
  } elsif ($l eq "</dl>") {
    &proc06("<br>");
    &exdent(4);
  } elsif ($l eq "<dir>") {
    &proc06("<br>");
    &indent(4);
  } elsif ($l eq "</dir>") {
    &proc06("<br>");
    &exdent(4);
  } elsif ($l eq "<dt>") {
    &proc06("<br>");
    &exdent(4);
  } elsif ($l eq "<em>") {
    &proc06("\023");
    $kill_ls = 1;
    return;
  } elsif ($l eq "</em>") {
    &proc06("\024");
  } elsif ($l eq "<hr>") {
    &proc06("<br>");
    &proc06("<hr 1>");
    # &proc06("-" x $screen_width);
    # &proc06("<br>");
  } elsif ($l =~ m|^<hr |) {
    # get size
    $size = "-";
    if ($l =~ m|size\=([0-9]+)[^0-9]|) {
      $size = $1;
      $size = ($size > 3) ? "=" : "-";
    }
    # get width
    $pct = 100;
    if ($l =~ m|width\=([0-9]+)\%|) {
      $pct = $1;
    }
    if ($pct < 100) {
      &proc06("<br>");
      $save_pre = $pre_f; $pre_f = 1;
      &proc06((" " x ($screen_width * (100 - $pct) / 200))
                        . ($size x ($pct * $screen_width / 100)));
      $pre_f = $save_pre;
      &proc06("<br>");
    } else {
      &proc06("<br>");
      &proc06(($size eq "-") ? "<hr 1>" : "<hr 2>");
    }
  } elsif ($l =~ m|^<li[ >]|) {
    &proc06("<br>");
    if ($in_ol) {
      $ol_level++;
      # Indent and exdent disabled for <li> because of this page:
      # http://www.macintouch.com/flatpanelimac.html
      # &indent(4);
      if ($ol_level == 1) {
        $li_num++;
        &proc06($li_num . ". ");
      } else {
        &proc06("* ");
      }
    } else {
      &proc06("* ");
    }
  } elsif ($l eq "</li>") {
    $ol_level--;
    # &exdent(4);
  } elsif ($l =~ m|^<ul[ >]|) {
    &proc06("<br>");
    &indent(4);
    $in_ul = 1;
    return;
  } elsif ($l eq "</ul>") {
    &exdent(4);
    &proc06("<br>");
    $in_ul = 0;
    return;
  } else {
    &proc06($l);
  }
}

# remove CR and LF within atoms, lowercase HTML tags, etc.
sub proc04
{
  local($l) = @_;
  local($l1, $l2, $l3);

  # handle view source
  if ($view_source == 4) {
    &pr2("$mk_pr2$l");
    return;
  }

  # Remove CR and LF within atoms
  if ($l =~ m/^<.*>$/) {
    $l =~ s/\001/ /g;
  }

  # Lowercase keywords of atoms
  if ($l =~ m|^<([a-zA-Z/]+)(.*)$|){
    $l1 = $1;
    $l2 = $2;
    $l1 =~ tr/A-Z/a-z/;
    $l = "<" . $l1 . $l2;
  } elsif (($this_gsc) && ($l =~ m/^(.*)($gsc_key)(.*)$/)) {
    $l1 = $1; $l2 = $2; $l3 = $3;
    &proc05($l1);
    &proc05($zm); # hot pink
    &proc05($l2);
    &proc05($zn); # normal
    $l = $l3;
  }

  &proc05($l);
}

# combine parts of atoms into single lines
sub proc03
{
  local($l) = @_;

  # handle view source
  if ($view_source == 3) {
    &pr2("$mk_pr2$l");
    return;
  }

  if ($l =~ m|<|) {
    if ($l =~ m|>|) {
      # self-contained line
      &proc04($prev . $l);
      $prev = "";
    } else {
      # beginning of an atom
      $prev .= $l;
    }
  } else {
    if ($l =~ m|>|) {
      # found the end of a command atom
      $prev .= " " . $l;
      &proc04($prev);
      $prev = "";
    } else {
      # neither
      if ($prev eq "" ) {
        &proc04($l);
      } else {
        # in the middle of a long one
        $prev .= " " . $l;
      }
    }
  }
}

# parse text into separate atoms
sub proc02
{
  local($l) = @_;
  local($l1, $l2);

  # handle view source
  if ($view_source == 2) {
    &pr2("$mk_pr2$l");
    return;
  }

  while (($spoolreal || $lof_going) && ($l ne "")) {
    if ($l =~ m|^([^<>]*)\>(.*)$|) {
      # got end part of an atom on the front
      $l1 = $1 . ">";
      $l2 = $2;
      &proc03($l1);
      $l = $l2;
    } elsif ($l =~ m|^([^<]+)(\<[a-zA-Z!/].*)$|) {
      # got plain text on the front, then an atom or part of an atom
      $l1 = $1;
      $l2 = $2;

      &proc03($l1);
      $l = $l2;
    } elsif ($l =~ m|^(\<[^>]*\>)(.*)$|) {
      # got a complete atom on the front
      $l1 = $1;
      $l2 = $2;

      &proc03($l1);
      $l = $l2;
    } else {
      # line is beginning part of an atom
      &proc03($l);
      $l = "";
    }
  }
}

# proc01 removes control characters (except newlines)
sub proc01
{
  local($l) = @_;
  local($l2);

  # handle view source
  if ($view_source == 1) {
    &pr2("$mk_pr2$l");
    return;
  }

  # This script uses pattern matching (like $l =~ m/^foo$/)
  # and for some reason the '$' in patterns treats newline characters
  # differently from all other characters. So, to avoid that problem
  # we convert newlines to \001, then convert \001 back to newlines
  # when we're ready to print.
  # First, wipe out control-A's
  $l =~ s/\001//g;
  # Then convert all standard and native newline and return characters to control-A
  $l =~ s/[\n\r\012\015]+/\001/g;
  # Then wipe out all other control characters except tab.
  $l =~ s/[\002-\010\012-\037]//g;
  # NOTE: not doing anything about \000 or \177 right now.

  foreach $l2 (split("\001", $l)) {
    &proc02($l2 . "\001") if ($l2 ne "");
  }
}

sub reset_pagedata
{
  # reset frames
  $cur_frame = 0;
  $num_frames = 0;
  undef @frame_url;
  # reset anchors
  $anc_num = 0;
  undef @anchors;
  undef @anc_base;
  undef @anc_text;
  undef %rev_anc;
  # reset forms
  $form_num = 0;
  undef @form_stack;
  # reset inputs (form fields)
  $inp_num = 0;
  if ($retain_form) {
    # don't change value, etc.
  } else {
    undef @input_value;
    undef @input_checked;
  }
  # But these will get reloaded during formatting anyway
  undef @input_form;
  undef @input_type;
  undef @input_name;
  # reset images
  $img_num = 0;
  $embed_num = 0;
  undef @img_src;
  undef @embed_src;
  undef @img_base;
  undef @embed_base;
  undef %rev_img;
  undef %rev_embed;
  # other resets
  $page_title = "";
  $in_ol = 0;
  $in_ul = 0;
  $in_anchor = 0;
  $more_linecount = 0;
}

# stream_n_format expects an input stream to be open on INFILE and
# an output to be open on SPOOLFILE. It copies INFILE to SPOOLFILE and
# formats the data to STDOUT through the HTML formatting pipeline
# starting with proc01.
sub stream_n_format
{
  local($load_url) = @_;

  $f_ovkey = "";
  $ignore = 0;
  $didbr = 1;
  $in_bold = 0;
  $bq_indent = 0;
  $ol_level = 0;
  $prev = "";
  if ($load_url =~ m|\.html$|) {
    $pre_f = 0;
  } elsif ($load_url =~ m|\.htm$|) {
    $pre_f = 0;
  } elsif ($load_url =~ m|/$|) {
    $pre_f = 1;
  } else {
    $pre_f = 1;
  }

  $isgif = 0;
  $lof_going = 1;
  while (($l = <INFILE>) && ($spoolreal || $lof_going)) {
    if ($isgif == 0) {
      if ($l =~ m|GIF8[79]a|) {
        $isgif = 1;
      } elsif ($l =~ m|<html|i) {
        $isgif = -1;
      }
    }
    print SPOOLFILE $l;
    if ($isgif <= 0) {
      if(!($justcache)) {
        &proc01($l);
      }
    }
  }
  print $g_normal;

  $pre_f = 0;
  $ignore = 0;
  $didbr = 0;
  $in_bold = 0;
  $in_anchor = 0;
  $bq_indent = 0;
  $prev = "";
}

# stream_live opens a live stream (HTTP) and formats it. It uses
# stream_n_format to do the actual formatting, and catches the ctrl-C
# (control-C) signal in case the user wants to interrupt a stalled
# transfer or stop a really long stream of data.
sub stream_live
{
  local($load_url) = @_;
  local($cmd, $stat1);

  $stat1 = 0;

  chdir $tmpdir;
  unlink "lynx_err";
  $load_url =~ s|'|'"'"'|g;

  if ($load_url =~ m|^file:|) {
    $load_url =~ s|^file:||;
    $load_url =~ s|^//localhost/|/|;
    $cmd = "cat '$load_url' |";
    $spoolreal = 0;
    $stat1 = 1;
  } else {
    $cmd = "bwget";
    if ($load_url =~ m/\?/) {
      if ($fmethod =~ m/post/i) {
        $cmd = "bwpost";
      }
    }
    $cmd = $cmd . " '$load_url' |";
    $spoolreal = 1;
  }

# this print is for debugging #mirad#
print "$cmd\n";

  open (INFILE, $cmd);
  $htps_path = "$tmpdir/http_spool.$$";
  open (SPOOLFILE, "> $htps_path");
  {
    local $SIG{INT} = sub { die "User aborted HTTP load\n" };
    eval {
      &stream_n_format($load_url);
    };
    if ($@ ne "") {
      print $red;
      print ($@);
      print $g_normal;
      $stat1 = 404;
    }
  }
  close INFILE;
  close SPOOLFILE;

  if ($stat1 == 0) {
    local($erf);
    $erf = "$tmpdir/bwget_result";
    open(ERROR, $erf);
    $stat1 = <ERROR>; chop $stat1;
    close ERROR;
    # remove it to avoid possibly throwing off the results of the next get
    unlink($erf);
  }

  return $stat1;
}

sub stream_cache
{
  my($load_url) = @_;
  my($cpath, $needcomp);

  $cpath = &cache_path($load_url, 1);
  # check for compressed.gz file
  # print "foo1 $cpath\n";
  if (-e $cpath) {
    open (INFILE, $cpath);
    $needcomp = 1;
  } else {
    open (INFILE, "cat $cpath.gz | zcat |");
    $needcomp = 0;
  }
  open (SPOOLFILE, "> /dev/null");
  $spoolreal = 0;

  {
    local $SIG{INT} = sub { die "User aborted HTTP load\n" };
    eval {
      &stream_n_format($load_url);
    };
    if ($@ ne "") {
      print $red;
      print ($@);
      print $g_normal;
      $stat1 = 404;
    }
  }

  close INFILE;
  close SPOOLFILE;
  # compress cache file if it isn't already compressed
  if ($needcomp) {
    system("gzip -f $cpath");
  }
}

sub cache_transfer
{
  local($from_url, $to_url) = @_;
  local($from_cp, $to_cp, $cmd1);

  if ($from_url ne $to_url) {
    $from_cp = &cache_path($from_url, 1);
    # need to check for compressed.gz files
    $to_cp = &cache_path($to_url, 1);

    if ($from_cp ne $to_cp) {
      &cache_mkdir($to_cp);
      if (-e $from_cp) {
        $cmd1 = "cp $from_cp $to_cp";
      } elsif (-e "$from_cp.gz") {
        $cmd1 = "cat $from_cp.gz | zcat > $to_cp";
      }
print "\n1 $cmd1\n";
      system($cmd1);
      # make destination into a compressed.gz file
      system("gzip -f $to_cp");
print "\n1 cache transfer $from_cp -> $to_cp for $to_url\n";
    }
  }
}

sub cache_exists
{
  local($load_url) = @_;
  local($cpath);

  $cpath = &cache_path($load_url, 1);
  if ($cpath eq "") {
    return 0;
  } elsif (-e $cpath) {
    return 1;
  } elsif (-e "$cpath.gz") {
    # exists as compressed.gz file
    return 1;
  }
  return 0;
}

sub cache_size
{
  local($load_url) = @_;
  local($cpath);

  $cpath = &cache_path($load_url, 1);
  if ($cpath eq "") {
    return 0;
  } elsif (-e $cpath) {
    return (-s $cpath);
  } elsif (-e "$cpath.gz") {
    # exists as compressed.gz file
    return (-s "$cpath.gz");
  }
  # else
  return 0;
}

# This routine isn't actually used -- yet
sub kill_cache
{
  local($load_url) = @_;
  local($cpath);

  $cpath = &cache_path($load_url, 1);
  if ($cpath eq "") {
    # nothing
  } elsif (-e $cpath) {
    unlink $cpath;
  } elsif (-e "$cpath.gz") {
    # exists as compressed.gz file
    unlink "$cpath.gz";
  }
}

sub lf1
{
  print "\n" if (!($justcache));
}

sub load_n_cache
{
  local($load_url) = @_;
  local($cpath, $cpokay, $stat2);
  local($load_live);

  $load_live = $prefer_live;

  # determine cache path for this URL (or none if the cache path
  # algorithm is confused)
  if ($load_url =~ m|^file:|) {
    $cpath = "";      # indicates no cache file
    $load_live = 1; # force flow through first part of big if-then
  } else {
    $cpath = &cache_path($load_url, 0);
    $lnc_url = $load_url;
    if ($cpath eq "") {
      $cpath = "/dev/null";
      $cpokay = 0;
    } else {
      &cache_mkdir($cpath);
      $cpath = "$cachedir/$cpath";
      $cpokay = 1;
    }
  }

  if ($load_live || ($proxy_reload)) {
    $stat2 = &stream_live($load_url);

    if ($isgif == 1) {
      &lf1();
      print "$magenta"."<GIF file, output suppressed>$g_normal\n";
    } else {
      &lf1();
    }
    # If we got a good file, update the cache. Else, load from the cache
    if ($stat2 == 1) {
      # It was a file:// url
      $stat2 = 200;
      &lf1();
      print "Status was $stat2.\n";
    } elsif (   ($stat2 == 200)
        || ($stat2 == 0)
    ) {
      if ($lnc_url ne $load_url) {
        $cpath = &cache_path($lnc_url, 1);
        &cache_mkdir($cpath);
      }
      # print "\n2 cp $htps_path $cpath\n";
      system("cp $htps_path $cpath");
      # compress it into a compressed.gz file
      system("gzip -f $cpath");
      # print "\n2 cache into $cpath for $lnc_url\n";
      &lf1();
      print "Status was $stat2.\n";
    } elsif ($cpokay && (-e $cpath)) {
      &lf1();
      &stream_cache($load_url);
      &lf1();
      print "Status was $stat2; loaded old version from cache at '$cpath'.\n";
    } elsif ($cpokay && (-e "$cpath.gz")) {
      &lf1();
      &stream_cache($load_url);
      &lf1();
      print "Status was $stat2; loaded old (compressed) from cache at '$cpath.gz'.\n";
    } else {
      &lf1();
      print "Status was $stat2; no cache copy available.\n";
    }
  } else {
    if ($cpokay && ((-e $cpath) || (-e "$cpath.gz"))) {
      if ($justcache) {
        $skipped_stream = 1;
      } else {
        &stream_cache($load_url);
        &lf1();
      }
      if ($skipped_stream) {
        print "$magenta"."Resource is a GIF file or already in cache.$g_normal\n";
      } elsif ($isgif == 1) {
        print "$magenta"."<GIF file, output suppressed>$g_normal\n";
      } else {
        if (-e $cpath) {
          print "$magenta"."Loaded old version from cache at '$cpath'.$g_normal\n";
        } else {
          print "$magenta"."Loaded old (compressed) from cache at '$cpath.gz'.$g_normal\n";
        }
      }
    } else {
      $stat2 = &stream_live($load_url);

      &lf1();
      if ($isgif == 1) {
        print "$magenta"."<GIF file, output suppressed>$g_normal\n";
      }
      # If we got a good file, update the cache.
      if (   ($stat2 == 200)
          || ($stat2 == 0)
      ) {
        if ($lnc_url ne $load_url) {
          # print "\n4 cp $htps_path $cpath\n";
          system("cp $htps_path $cpath");
          # compress it into a compressed.gz file
          system("gzip -f $cpath");
          # print "\n4 cache into $cpath for $load_url\n";

          $cpath = &cache_path($lnc_url, 1);
          &cache_mkdir($cpath);
        }
        # print "\n3 cp $htps_path $cpath\n";
        system("cp $htps_path $cpath");
        # compress it into a compressed.gz file
        system("gzip -f $cpath");
        # print "\n3 cache into $cpath for $lnc_url\n";
        &lf1();
        print "$green"."No cache copy available; Live status was $stat2.$g_normal\n";
      } else {
        &lf1();
        print "$red"
          ."No cache copy available; Live status was $stat2.$g_normal\n";
      }
    }
  }

  return $stat2;
}

sub dopage
{
  local($status);

  $goodload = 0;

  open (OUTFILE, "> $outname");
  open (ASCIIFILE, "> $litname");

  $load_url = $cur_url;
  print ASCIIFILE ("\n" . ("=-" x 38) . "=\n");
  print ASCIIFILE " URL: $load_url\n";
  print ASCIIFILE ("Date: " . localtime() . "\n\n");

  open (RHTFFILE, "> $rhtfname");
  print RHTFFILE ("MIRA output\ntcc[MIRA output]\n\nOriginal URL: $load_url\n` Date: " . localtime() . "\n\n");

  &reset_pagedata();

frame_loop: ;
  $status = &load_n_cache($load_url);
  if (($status == 0) || ($status == 200)) {
    if ($isgif <= 0) {
      $goodload = 1;
    }
  }

  # if we encountered any (more) frames, loop through them.
  if ($cur_frame < $num_frames) {
    &proc01("<br>");
    $cur_frame += 1;
    $load_url = $frame_url[$cur_frame];
    print $g_bold . "frame:$yellow$load_url$g_normal\n";
    goto frame_loop;
  }

  # we're done with the whole view now. Go ahead and close output.
  close OUTFILE;
  close ASCIIFILE;
  close RHTFFILE;
  system ("cat $outname | fold | $pager -r -X") if ($of_more);
}

sub encode_field
{
  local($t) = @_;
  local($rv);

  $rv = $t;
  $rv =~ s|%|%25|g;

  $rv =~ s| |%20|g;
  $rv =~ s|"|%22|g; # "
  $rv =~ s|&|%26|g;
  $rv =~ s|'|%27|g;
  $rv =~ s|\+|%2B|g;
  $rv =~ s|=|%3D|g;
  $rv =~ s|\?|%3F|g;
  $rv =~ s|\\|%5C|g;

  return $rv;
}

sub is_audio
{
  local($url) = @_;

  return (($url =~ m|.au$|i)
	|| ($url =~ m|.ram$|i)
    );
}

sub is_graphic
{
  local($url) = @_;

  return (($url =~ m|.jpg$|i)
	|| ($url =~ m|.gif$|i)
	);
}

sub is_pdf
{
  local($url) = @_;

  return ($url =~ m|.pdf$|i);
}

sub is_email
{
  local($url) = @_;

  print "$url\n";
  return ($url =~ m|^mailto:|i);
}

sub is_extern
{
  local($url) = @_;

  if ($url =~ m|\?|) {
    return 0;
  } else {
    return ( &is_graphic($url)
		|| &is_audio($url)
		|| &is_pdf($url));
  }
}

sub export_gh
{
  print "Sorting global history...";
  open (GHFILE, "> $miradir/g_hist");
  $oldfh = select(GHFILE); $| = 1; select($oldfh);
  $i = $ghp - 10000;
  $i = 1 if ($i < 1);
  for (; $i <= $ghp; $i++) {
    print GHFILE ($global_history[$i] . "\n");
  }
  close GHFILE;
  print "done.\n";
}

# Saves the current state of the "local" history (the forward-back stack)
sub export_lh
{
  open (LHFILE, "> $miradir/l_hist");
  $oldfh = select(LHFILE); $| = 1; select($oldfh);
  print LHFILE "$hist_ptr\n";
  for ($i=1; $i <= $hist_top; $i++) {
    print LHFILE ($history[$i] . "\n");
  }
  close LHFILE;
}

# Loads the "local" history stack from the save file.
sub import_lh
{
  local($url, $i);
  if (open (LHFILE, "$miradir/l_hist")) {
    $hist_ptr = <LHFILE>; chop $hist_ptr;
    $i = 0;
    while ($url = <LHFILE>) {
      chop $url;
      $i++;
      $history[$i] = $url;
    }
    $hist_top = $i;
    close LHFILE;
    return 0;
  } else {
    $hist_ptr = 0;
    return 1;
  }
}

# This goes through and attempts to reload every page you've visited (as
# given by the global history file)
sub reload_gh
{
  local($url, $status, $sz, $i, $going);
  local($save_pl);

  $save_pl = $prefer_live;
  $prefer_live = 1;
  open (GHFILE, "$miradir/g_hist");
  $going = 1; $i = 0;
  while (($url = <GHFILE>) && $going) {
    chop $url;
    # don't bother to reload file:// and other non-HTTP stuff
    if ($url =~ m/http/) {
#    if ($url =~ m/lambda/) {
      $sz = &cache_size($url);
      if ($sz < 100) {
        print "\n\n\n\n$sz$g_bold ";
        &reset_pagedata();
        $status = &load_n_cache($url);
        print "$g_normal";
        if ($status == 200) {
          $i++;
        }
      }
    }
  }
  close GHFILE;
  &reset_pagedata();
  print "\n$yellow$i pages reloaded successfully.$g_normal\n";
  $prefer_live = $save_pl;
}

# Search the contents of a singla cache file (given by a URL), for a pattern.
sub search_cache
{
  local($load_url, $key) = @_;
  local($cpath, $rv, $l, $i, $fnd, $needcomp);

  # determine cache path for this URL (or none if the cache path
  # algorithm is confused)
  $cpath = &cache_path($load_url, 1);

  if ($cpath eq "") {
print "cpath null\n";
    return 0;
  }

  if (-f $cpath) {
    # okay to continue
    open(SC_IN, $cpath);
    $needcomp = 1;
  } elsif (-f "$cpath.gz") {
    # it's a compressed.gz file; okay to continue
    open(SC_IN, "cat $cpath.gz | zcat |");
    $needcomp = 0;
  } else {
    # no file
print "no file $cpath\n";
    return 0;
  }

  # Search for the pattern
  $rv = 0; $i=0;
  while (($rv == 0) && ($l = <SC_IN>)) {
    chop $l;
    $i++;
    if ($matchcase) {
      $fnd = ($l =~ m|$key|);
    } else {
      $fnd = ($l =~ m|$key|i);
    }
    if ($fnd) {
      $rv = 1;
    }
  }
  close SC_IN;

  # compress it if it isn't already compressed
  if ($needcomp) {
    system("gzip -f $cpath");
  }

  return $rv;
}

# This searches all pages in reverse order (order given by the global
# history file)
sub search_gh
{
  local($key, $passcount, $i_ghp) = @_;
  local($url, $status, $sz, $i, $going);
  local($gp, $found, $rv, $finds);

  $gsc_key = $key;
  $gp = $ghp;

  # allow caller to specify initial history item number
  if ($i_ghp > 0) {
    $gp = $i_ghp;
  }

  $going = 1; $i = 0; $rv = $url; $finds = 0;

  {
    local $SIG{INT} = sub {
               die ($red . "Global search by content aborted\n" . $g_normal)
                          };
    eval {
      while(($gp > 0) && ($going)) {
        $url = $global_history[$gp];
        $sz = &cache_size($url);
        print "$gp $url";
        if ($sz > 0) {
          $found = &search_cache($url, $key);
          $i++;
          if ($found) {
            $finds++;
            if ($finds >= $passcount) {
              $going = 0;
              $rv = $url;
              $sgh_ghp = $gp;
            } else {
              print " (skipped)";
            }
          }
        }
        print "\n";

        $gp--;
      }
    };
    if ($@ ne "") {
      print $red;
      print ($@);
      print $g_normal;
    }
  }

  &reset_pagedata();
  print "$yellow$i pages searched.$g_normal\n";
  return $rv;
}

# There are two "global history" files. One is g_log, which always grows,
# and is a simple chronological log of all URLs visited. The other is g_hist,
# and contains 10000 lines plus all the URLs visited in the latest session.
# When MIRA is launched, g_hist is loaded and sorted by the append_gh
# mechanism (see below) and then written back out. After that the file is
# just appended (the idea being that it will be sorted and duplicates removed
# the next time you start MIRA.) So, the first portion of g_hist contains
# unique URLs and the remainder is a chronological log of the current session.
#
# If you run multiple MIRA sessions simultaneously, both files g_log and
# g_hist will be updated by all running MIRA sessions. Thus, the global
# history will have URLs from different sessions interleaved with each other.
sub log_gh
{
  local($l) = @_;

  open (GHFILE, ">> $miradir/g_hist");
  $oldfh = select(GHFILE); $| = 1; select($oldfh);
  print GHFILE ($l . "\n");
  close GHFILE;

  open (GHFILE, ">> $miradir/g_log");
  $oldfh = select(GHFILE); $| = 1; select($oldfh);
  print GHFILE ($l . " " . localtime() . "\n");
  close GHFILE;
}

# This routine maintains the global_history array and its inverse, visited{}.
# When the user visits a URL that he has visited before, the URL gets shifted to
# the end of the global_history, and lots of other entries shift back one. At
# the same time the inverse array visited{} has to be updated to match.
sub append_gh
{
  local($url) = @_;
  local($v,$i);
  local($debug);
  $debug = 0; # if ($url =~ m/keyboard/) { $debug = 1; }

  print "\nappend_gh |$url|" if ($debug);
  $url = &munge1($url);
  $url = &url_base($url, 0);
  $v = $visited{$url};
  print " $v" if ($debug);
  if ($v) {
    # Already visited, so don't append -- but, move this URL to
    # the end of the global history.
    for ($i = $v; $i < $ghp; $i++) {
      $global_history[$i] = $global_history[$i+1];
      $visited{$global_history[$i]} = $i;
    }
    $global_history[$ghp] = $url;
    $visited{$url} = $ghp;
  } else {
    $ghp++;
    $global_history[$ghp] = $url;
    $visited{$url} = $ghp;
  }
  print " : $ghp\n" if ($debug);
}

# Loads the global history array from the global history file.
sub import_gh
{
  local($c1);

  print "Reading global history...";
  if (open (GHFILE, "$miradir/g_hist")) {
    $c1 = 0;
    while (<GHFILE>) {
      $l = $_;
      chop $l;
      while ($l =~ s/ .*$//) {
        # just keep going
      }
      &append_gh($l);
      $c1++;
      if ($c1 > 100) {
        print ".";
        $c1 -= 100;
      }
    }
    close GHFILE;
  }
  print " done.\n";
}

# Downloads a file and saves it
sub dl_file
{
  local($url) = @_;
  local($dest);

  # print "$url\n";
  $dest = "$dldir/dl.bin";
  if ($url=~m|/([^/&?]+)$|) {
    $dest = "$dldir/$1";
  }
  system("bwget '$url' > $dest");
  return $dest;
}

# Determines if Netscape is already running.
sub ns_running
{
  local($l, $f);

  open(NSR_STREAM, "ps a |");
  $f = 0;
  while($l = <NSR_STREAM>) {
    if ($l =~ m/netscape-communicator/) {
      $f = 1;
    }
  }
  close NSR_STREAM;
  return $f;
}

# Runs external viewer on an image
sub ev_image
{
  local($url) = @_;

  # print "ev_image $url\n";

  if ($url =~ m/^(.*)\.([a-z]+)$/) {
    $suffix = $2;
  } else {
    $suffix = "gif";
  }
  system("bwget '$url' > t_image.$suffix");
  if ($eviewer_img eq "open") {
    system($eviewer_img, "t_image.$suffix");
  } else {
    system("$eviewer_img t_image.$suffix $imloc &");
  }
}

# Runs external viewer on an image
sub ev_pdf
{
  my($url) = @_;

  # print "ev_pdf($url)\n";

  chdir $tmpdir;
  if ($url =~ m/^(.*)\.([a-z]+)$/) {
    $suffix = $2;
  } else {
    $suffix = "pdf";
  }
  # print "bwget '$url' > t_image.$suffix\n";
  system("bwget '$url' > t_image.$suffix");
  # print "$eviewer_pdf 't_image.$suffix' $pdf_imloc &";
  system("$eviewer_pdf t_image.$suffix $pdf_`imloc &");
}

# Runs external viewer on an audio file
sub ev_audio
{
  local($url) = @_;

  if ($url =~ m/^(.*)\.([a-z]+)$/) {
    $suffix = $2;
  } else {
    $suffix = "au";
  }
  system("bwget '$url' > t_image.$suffix");
  system("$eviewer_aud t_image.$suffix &");
}

# Runs external viewer on a URL (treating the data as a raw text file)
# in most cases the text has already been loaded into the cache, so we can
# just launch the eviewer on the existing cache file. But, if there isn't
# a file in the cache then that means one of two things:
#  - Either there was an error trying to load the file into the cache and
#    there will probably be an error if we try it again, or
#  - We're being called on a file that the user didn't load explicitly,
#    so probably the file we're loading doesn't belong in the cache (at
#    least not right now.)
sub ev_text
{
  local($url) = @_;
  local($cpath, $suffix, $launchit, $getcmd);

  $cpath = &cache_path($url, 1);
  $launchit = "$eviewer_vse $cpath";
  if (-f $cpath) {
    # Normal case -- a page we've just loaded and already in the cache.
    # everything's all set
  } elsif (-f "$cpath.gz") {
    # page is a compressed.gz file in the cache
    system("gunzip $cpath");

    # The file will be left uncompressed -- don't know how to "catch"
    # this and recompress.
  } else {
    # Not in cache. Either there was an error (like URL to path conversion)
    # so there's no point trying to load it into the cache again; or we're
    # being called in a "temporary" or "subroutine" context where we don't
    # want to change the cache.
    chdir $tmpdir;
    if ($url =~ m/^(.*)\.([a-z]+)$/) {
      $suffix = $2;
    } else {
      $suffix = "txt";
    }
    $getcmd = "bwget '$url' > t_image.$suffix";
    print "$getcmd\n"; system($getcmd);
    $launchit = "$eviewer_vse t_image.$suffix";
  }

  print "$launchit\n";
  system($launchit);
}

sub print_input
{
  local($i) = @_;

  print $inverse . "i$i$g_normal"
    . " F" . $input_form[$i]
    . ' "' . $input_name[$i] . '"'
    . " $yellow" . $input_type[$i] . $g_normal
    . ": " . $input_value[$i]
    . $input_checked[$i]
    . "\n";
}

sub print_email
{
  local($adr, $subject, $path) = @_;
  local($address, $cmd2, $cmd);

  print "Mail to <mrobu>:";
  $address = <>; chop $address; $address = "mrobu" if ($address eq "");
  $subject =~ s/[^-.a-zA-Z0-9]//g;
  if ($subject =~ m|^(.{50})|) {
    $subject = $1;
  }
  $em_seq++;
  open (FOO, "> $tmpdir/spool.$em_seq");
  print FOO " Title: $subject\n";
  print FOO "   URL: $cur_url\n";
  print FOO "  Date: " . localtime() . "\n";
  print FOO " \n";
  close FOO;
  if (-f $path) {
    $cmd2 = "cat $path >> $tmpdir/spool.$em_seq";
  } elsif (-f "$path.gz") {
    $cmd2 = "cat $path.gz | zcat >> $tmpdir/spool.$em_seq";
  } else {
    print "\nprint_email: no file or .gz file!\n";
    exit 0;
  }
  system $cmd2;
# $cmd = "mail1 $address $tmpdir/spool.$em_seq $subject 2>&1 > /dev/null &";
  $cmd = "mail1 $address $tmpdir/spool.$em_seq $subject 2>&1 &";
  print "Sending...\n";
  print "$cmd\n";
  system $cmd;
}

sub do_email
{
  local($url) = @_;
  local($address, $subject, $l, $cmd);

  if ($url =~ m|^mailto:(.+)$|) {
    $address = $1;
    print "\n     To: $address\nSubject: ";
    $subject = <>; chop $subject;
    $subject =~ s/[']//g;
    $em_seq++;
    open (FOO, "> $tmpdir/spool.$em_seq");
    $l = "";
    print "Enter text, end with '.':\n";
    while ($l ne ".") {
      $l = <>;
      print FOO $l;
      chop $l;
    }
    close FOO;
#   $cmd = "mail1 $address $tmpdir/spool.$em_seq $subject 2>&1 > /dev/null &";
    $cmd = "mail1 $address $tmpdir/spool.$em_seq $subject 2>&1 &";
    print "Sending...\n";
    print "$cmd\n";
    system $cmd;
  } else {
    print "\nCan't recognize address in email URL $url\n";
  }
}

sub grep_bookmarks
{
  local ($pat) = @_;
  local ($l, $url, $href, $title);

  $url = $bm_url;

  open(BM_IN, $bm_path);
  while ($l = <BM_IN>) {
    chop $l;
    if ($l =~ m|^<a href=\"([^\"]+)\">(.*)</a>|) {  # "
      $href = $1;
      $title = $2;
      if ($title =~ m|$pat|) {
        $url = $href;
      }
    }
  }
  close BM_IN;

  return $url;
}

sub makehomedir
{
  local($dir) = @_;

  $dir = $homedir . "/" . $dir;

  if (-d $dir) {
    # all set
  } elsif (-e $dir) {
    print "
mira: Need to make a directory '$dir', but there's a file there now.\n";
    exit 1;
  } else {
    mkdir($dir, 0755);
  }

  return $dir;
}

sub check_radio
{
  local($inum) = @_;
  local($iname, $n);

  # Uncheck all items with same name as selected item
  $iname = $input_name[$inum];
  for($n=0; $n<$inp_num; $n++) {
    if ($input_name[$n] eq $iname) {
      $input_checked[$n] = " ";
    }
  }

  # check selected item
  $input_checked[$inum] = "*";
}

sub install_str1
{
  local($cmd);

  chdir $miradir;
  if (-e "mira-str1.c") {
    # here we would check to see if it's changed
  } else {
    if (-e "$bindir/mira-str1") {
      # mira-str1 exists but no mira-str1.c -- recompile to make sure
      unlink("$bindir/mira-str1");
    }

    print "Building and installing #mira-str1#...\n";

    # create the source file
    open(MIRAD, "> mira-str1.c");
    print MIRAD $str1_c_text;
    close MIRAD;

    # build and put result in bin directory
    $cmd = "gcc -o mira-str1 mira-str1.c";
    print "  $cmd\n"; system($cmd);
    system("mv mira-str1 $bindir");
  }
}

$k_copyleft_url = "http://www.gnu.org/copyleft";

$| = 1;

$savetodir = `pwd`; chomp $savetodir;

chdir;

$pager = "less";
$eviewer_html = "arena -cm"; # old viewer
$eviewer_html = "netscape";
$eviewer_txt = "lynx -child";

if (-x "/usr/bin/kvt") {
  $eviewer_src = "/usr/bin/kvt -e emcs-nox";
} elsif (-x "/opt/kde/bin/kvt") {
  $eviewer_src = "/opt/kde/bin/kvt -e emcs-nox";
} elsif (-x "/usr/bin/gnome-terminal") {
  $eviewer_src = "/usr/bin/gnome-terminal -x emcs-nox";
} elsif (-x "/usr/bin/xterm") {
  $eviewer_src = "xterm -e emcs-nox";
} else {
  $eviewer_src = "emacs";
}

$eviewer_vse = "emacs";

if ($eviewer_img = "/usr/X11R6/bin/display", (-x $eviewer_img)) {
} elsif ($eviewer_img = "/usr/X11R6/bin/xv", (-x $eviewer_img)) {
} else {
  $eviewer_img = "open";
}

if (&ismac()) {
  $eviewer_pdf = "/usr/bin/open";
  $pdf_imloc = "";
} else {
  $eviewer_pdf = "gv";
  $pdf_imloc = "'-ge +20+20'";
}
$eviewer_aud = "xanim";
  $imloc = "-ge +20+20";
$prefer_live = 0;

&init_statics();

$clear_screen = 0;
$proxy_reload = 0;

chdir;
$homedir = `pwd`; chop $homedir;
$bindir = &makehomedir("bin");
$tmpdir = &makehomedir("tmp");
$dldir = &makehomedir("tmp/dl");
$miradir = &makehomedir("mira");
$cachedir = &makehomedir("mira/cache");
$mddir = &makehomedir("mira/mirad");

chdir $bindir;
if (-e "mirad") {
  # here we would check to see if it's changed
} else {
  print "Installing #mirad#...\n";
  open(MIRAD, "> mirad");
  print MIRAD $mirad_text;
  close MIRAD;
  system("chmod +x mirad");
}

&install_str1();

chdir $tmpdir;

$homepage_url = &munge1("file:$homedir/local.html");
$bm_path = "$miradir/mira-bookmarks.html";
$bm_url = "file:$bm_path";

$vl_expire = 0;

$justcache = 0;
$arg1 = shift;
if ($arg1 =~ m/^-[-]?h(elp)?$/) {
  print $help;
  exit(0);
} elsif ($arg1 eq "-c") {
  $justcache = 1;
  $init_url = shift;
} else {
  $init_url = $arg1;
}

if ($justcache) {
  $addstack = 1;
  $cur_url = $init_url;
} else {
  &import_gh();
  &export_gh();

  if ($addstack = &import_lh()) {
    $cur_url = $homepage_url;
    print "Starting from your home page '$cur_url'...\n";
  } elsif (($cur_url = $init_url) ne "") {
    $cur_url = &munge2($cur_url);
    print "Starting at '$cur_url'...\n";
    $addstack = 1;
  } else {
    $cur_url = $history[$hist_ptr];
    print "Restoring latest MIRA $g_bold"."f/b$g_normal stack.\n";
    print "Returning to '$cur_url'...\n";
  }
}

$going = 1;
$skip_to_input = 0;
while ($going) {
  if ($justcache) {
    # we'll quit right away
    $going = 0;
  }

  if (!($skip_to_input)) {
    if ($addstack) {
      $hist_ptr += 1;
      $history[$hist_ptr] = $cur_url;
      $hist_top = $hist_ptr;
    }
    if ($justcache == 0) {
      &export_lh();
    }

    if ($justcache) {
      $screen_size = " 24 80 ";
    } else {
      # measure the screen (might have changed since last time)
      $screen_size = " " . `stty size`;
      $old_screen_size = $screen_size;
      chop $screen_size;
      $screen_size .= " 0 0 ";
    }
    $screen_size =~ m| ([0-9]+) +([0-9]+) |;
    $screen_height = $1; $screen_height = 24 if ($screen_height < 10);
    $screen_width = $2; $screen_width = 80 if ($screen_width < 20);
    $screen_height -= 3;
    $screen_width -= 1;

    if ($clear_screen) {
      print $esc . "[H" . $esc . "[J" 
    } else {
      print "\n\n\n" if (!($justcache));
    }
    if(!($justcache)){
      print "$cur_url\n";
      print ("=_" x ($screen_width / 2));
      print "\n";
    }
    $outname = "$tmpdir/t_main";
    $litname = "$tmpdir/a_main";
    $rhtfname = "$tmpdir/r_main";

    $this_url = ""; $orig_url = "";
    if (&is_extern($cur_url)) {
      print "<Current URL is an external data type.>\n";
    } else {
      &dopage();
    }
    if ($this_url ne "") {
      # need to transfer the data we loaded into another cache entry.
#      &cache_transfer($cur_url, $this_url);

      # If they're different, record in global history
      if ($addstack && ($this_url ne $cur_url)) {
        if ($glr_mode) {
          # don't adjust global history or append to log.
        } else {
          &append_gh($cur_url);
          if (($justcache == 0) || ($goodload)) {
            &log_gh($cur_url);
          }
        }
      }

      # update URL for the purpose of executing commands
      if ($this_url ne $cur_url) {
        $orig_url = $cur_url;
      }
      $cur_url = $this_url;

      # and retroactively fix the history.
      $history[$hist_ptr] = $cur_url;
    }

    # record in global history
    if ($addstack) {
      if ($glr_mode) {
        # don't adjust global history or append to log.
      } else {
        &append_gh($cur_url);
        if (($justcache == 0) || ($goodload)) {
          &log_gh($cur_url);
        }
      }
    }

    if ($justcache) {
      if ($htps_path ne "") {
        unlink($htps_path);
      }
      exit 0;
    }

    # reset flags that are set on a per-command basis
    $proxy_reload = 0;
    $of_more = 0;
    $view_source = 0;
  }

# input1: ;
  $skip_to_input = 0;
  $defer_cmds =~ s/^;+//;
  if ($defer_cmds eq "") {
    # prepare for user input
    $page_title =~ s/[\001-\010\013-\014\016-\037]//g;
    print "$g_bold$cur_url  $yellow  $page_title$g_normal\n";
    if ($orig_url ne "") {
      print " ( auto-redirected from: $inverse"."ro$g_normal$g_bold $orig_url$g_normal )\n";
    }
    if ($default_command =~ m/([\001-\037])/) {
      $cc = $1; $cc =~ tr/\001-\037/\041-\077/;
      print "Control-$cc character in default command.\n";
      exit 1;
    }
    if ($default_command ne "") {
      print "[" . $default_command . "] ";
    }
    if ($glr_mode == 1) {
      print "{GLR} ";
    } elsif ($glr_mode == 2) {
      print "{GHR (gh $glr_index)} ";
    }

    print ">";
    $cmd = <>;

    $cmd = &fixerase($cmd);

    # Disabled the cursor-up because it's confusing. -19990114
    # print $up;
    chop $cmd;
    $cmd =~ s/^ +//;
    $cmd =~ s/[\001-\037]//g;

    if ($cmd eq "!!") {
      $cmd = $bangbang;
      print ">$cmd\n";
    } else {
      $bangbang = $cmd;
    }
  } else {
    $cmd = $defer_cmds;
    $defer_cmds = "";
  }
  if ($cmd =~ m|^([^;]*);(.*)$|) {
    $cmd = $1;
    $defer_cmds = $2;
  }
  if ($cmd eq "") {
    $cmd = $default_command;
  }
  $default_command = "";

  # assume form will get wiped, unless command says otherwise.
  $retain_form = 0;

  # Reset states per command-execution go here
  $last_gsc = $this_gsc;
  $this_gsc = 0;

  # Parse the command and arguments
  if ($cmd eq "") {
    $skip_to_input = 1;
  } elsif ($cmd =~ m|^a (.*)$|) { #------------------------------------------ a
    $glr_mode = 0;
    $new_url = $1;
    $new_url = &apply_href($cur_url, $new_url, 0);
    $new_url = &munge1($new_url);
    $addstack = 1;
  } elsif ("$cmd 1 " =~ m|^b ([0-9]+) |) { #--------------------------------- b
    $howmany = $1;
    if ($glr_mode) {
      $glr_index -= $howmany;
      if ($glr_index < 0) {
        $glr_index = 0;
      }
	  if ($glr_mode == 1) {
        # %%% read appropriate line of logfile.
      } else {
        $new_url = $global_history[$glr_index];
      }
    } else {
      if ($hist_ptr > $howmany) {
        $hist_ptr -= $howmany;
        $new_url = $history[$hist_ptr];
        $addstack = 0;
      } elsif ($hist_ptr > 1) {
        print "Can't go back $howmany, going to first page instead...\n";
        $hist_ptr = 1;
        $new_url = $history[$hist_ptr];
        $addstack = 0;
      } else {
        print "You are already at the first page.\n";
        $skip_to_input = 1;
      }
    }
    $default_command = "b";
  } elsif ($cmd eq "bh") { #------------------------------------------------ bh
    $gotit = 0;
    $i = $hist_ptr;
    while (($gotit == 0) && ($i > 1)) {
      $i--;
      $u = $history[$i];
      if (&url_host($u) eq &url_host($cur_url)) {
        # keep looking
      } else {
        $gotit = 1;
      }
    }
    if ($gotit) {
      $hist_ptr = $i;
      $new_url = $history[$hist_ptr];
      $addstack = 0;
    } else {
      print "Can't go back to previous host because all pages on the stack are from the\ncurrent host.\n";
      $skip_to_input = 1;
    }
  } elsif ($cmd =~ m|^bm *(.*)$|) { #--------------------------------------- bm
    $t = $1;
    if ($t eq "") {
      $t = $page_title;
      if ($t =~ m|^ *$|) {
        $t = &url_host($cur_url);
      }
    }
    open (BOOKMARKS, ">> $miradir/mira-bookmarks.html");
    print BOOKMARKS "<a href=" . '"' . $cur_url . '"' . "> $t </a><br>\n";
    close BOOKMARKS;
    print "Added bookmark " . '"' . $t . '"' . " to URL $cur_url\n";
    $skip_to_input = 1;
  } elsif ($cmd =~ m|dl ([0-9]+) *$|) { #---------------------------------- dl
    # download data linked to by given anchor number
    $cmd = $1;
    $dl_url = $anchors[$cmd];
    $dl_url = &apply_href($anc_base[$cmd], $dl_url, 1);
    print "Attempting to download '$dl_url' into $dldir...\n";
    $dl_url = &dl_file($dl_url);
    print "Binary is now in $dl_url. Copy it elsewhere before downloading another.\n";
    $skip_to_input = 1;
  } elsif ($cmd =~ m|dl \"([^\"]+)\"|) { # " - - - - - - - - - - - - - - -  dl
    # download data linked to by an href
    $dl_url = $1;
    $dl_url = &apply_href($cur_url, $dl_url, 1);
    print "Attempting to download '$dl_url' into $dldir...\n";
    $dl_url = &dl_file($dl_url);
    print "Binary is now in $dl_url. Copy it elsewhere before downloading another.\n";
    $skip_to_input = 1;
  } elsif ($cmd =~ m|dl em ([0-9]+) *$|) { #---------------------------- dl em
    # download an EMBED (plugin source data)
    $dl_url = &apply_href($embed_base[$1], $embed_src[$1], 1);
    $dl_url = &apply_href($cur_url, $dl_url, 1);
    print "Attempting to download '$dl_url' into $dldir...\n";
    $dl_url = &dl_file($dl_url);
    print "Binary is now in $dl_url. Copy it elsewhere before downloading another.\n";
    $skip_to_input = 1;
  } elsif ($cmd =~ m|dl im ([0-9]+) *$|) { #---------------------------- dl im
    # download an image from current page
    $dl_url = &apply_href($img_base[$1], $img_src[$1], 1);
    $dl_url = &apply_href($cur_url, $dl_url, 1);
    print "Attempting to download '$dl_url' into $dldir...\n";
    $dl_url = &dl_file($dl_url);
    print "Binary is now in $dl_url. Copy it elsewhere before downloading another.\n";
    $skip_to_input = 1;
  } elsif ($cmd eq "ev") { #----------------------------------------------- ev
    # launch external viewer
    # The remote signaling technique was learned from here:
    # http://home.netscape.com/newsref/std/x-remote.html
    if (&ns_running()) {
      $launch_cmd = "$eviewer_html -remote 'openURL($cur_url)'";
    } else {
      $launch_cmd = "$eviewer_html '$cur_url' >/dev/null 2>&1 &";
    }
    print "$launch_cmd\n";
    system $launch_cmd;
    $skip_to_input = 1;
  } elsif ($cmd eq "eva") { #--------------------------------------------- eva
    # launch ASCII external viewer
    $launch_cmd = "$eviewer_src '$litname'";
    print "$launch_cmd\n";
    system $launch_cmd;
    $skip_to_input = 1;
  } elsif ($cmd eq "evt") { #--------------------------------------------- evt
    # launch external viewer
    $launch_cmd = "$eviewer_txt '$cur_url'";
    print "$launch_cmd\n";
    system $launch_cmd;
    $skip_to_input = 1;
  } elsif ($cmd eq "exit") { #-------------------------------------------- exit
    # quit
    $going = 0;
  } elsif ("$cmd 1 " =~ m|^f +([0-9]+) |) { #-------------------------------- f
    $howmany = $1;
    if ($glr_mode) {
      $glr_index += $howmany;
	  if ($glr_mode == 1) {
        # %%% increment and compare to length of logfile, error if at end
        # %%% read appropriate line of logfile.
      } else {
        if ($glr_index > $ghp) {
          $glr_index = $ghp;
          print "Can't go forward anymore, you're at the frontier of history.\n";
          $skip_to_input = 1;
        }
        if (!($skip_to_input)) {
          $new_url = $global_history[$glr_index];
        }
      }
    } else {
      if ($hist_ptr + $howmany <= $hist_top) {
        $hist_ptr += $howmany;
        $new_url = $history[$hist_ptr];
        $addstack = 0;
      } elsif ($hist_ptr < $hist_top) {
        print "Can't go forwards $howmany, going to frontier...\n";
        $hist_ptr = $hist_top;
        $new_url = $history[$hist_ptr];
        $addstack = 0;
      } else {
        print "Can't go forward from here -- you're at the frontier.\n";
        $skip_to_input = 1;
      }
    }
    $default_command = "f";
  } elsif ($cmd =~ m|^g +(.*)$|) { #----------------------------------------- g
    $glr_mode = 0;
    $new_url = $1;
    $new_url = &munge2($new_url);
    $addstack = 1;
  } elsif ($cmd =~ m|^ga +(.+)$|) { #--------------------------------------- ga
    # go to anchor containing text
    $a = $1;
    $match = 0;
    for ($n=1; ($n <= $anc_num) && (! $match); $n++) {
      if ($anc_text[$n] =~ m|$a|) {
        $match = 1;
        $glr_mode = 0;
        $new_url = &apply_href($anc_base[$n], $anchors[$n], 1);
        $new_url = &munge1($new_url);
        $addstack = 1;
      }
    }
    if ($match) {
      # we already set up everything
    } else {
      print "No anchor matching '$cmd'. Type $g_bold"."sa$g_normal to see a\nlist of valid anchors.\n";
      $skip_to_input = 1;
    }
  } elsif ($cmd =~ m|^gb +(.*)$|) { #--------------------------------------- gb
    $glr_mode = 0;
    $new_url = $1;
    $new_url = &grep_bookmarks($new_url);
    $addstack = 1;
  } elsif ($cmd eq "gb") { #- - - - - - - - - - - - - - - - - - - - - - - -  gb
    $glr_mode = 0;
    $new_url = $bm_url;
    $addstack = 1;
  } elsif ($cmd eq "gh") { #------------------------------------------------ gh
    $i = $ghp - 20;
    $i = 1 if ($i < 1);
    for (; $i <= $ghp; $i++) {
      print $inverse . "gh $i$g_normal $global_history[$i]\n";
    }
    print "(more...)\n" if ($i > 21);
    $i = $ghp - 40; $i = 1 if ($i < 1);
    $default_command = "gh.$i";
    $skip_to_input = 1;
  } elsif ($cmd =~ m|^gh\.([0-9]+)$|) { #- - - - - - - - - - - - - - - - - - gh
    $i = $1;
    if ($i > $ghp) {
      print "There is no global history item $i. Type$yellow gh$g_normal to see the last 20 items.\n";
    } else {
      for ($j=0; $j < 20; $j++) {
        print $inverse . "gh $i$g_normal $global_history[$i]\n";
        $i++;
      }
      print "(more...)\n" if ($i > 21);
      $i -= 40; $i = 1 if ($i < 1);
      $default_command = "gh.$i";
    }
    $skip_to_input = 1;
  } elsif ($cmd =~ m|^gh ([0-9]+)$|) { #- - - - - - - - - - - - - - - - - -  gh
    # go to an entry in global history
    $i = $1;
    if (($i > 0) && ($i <= $ghp)) {
      $new_url = $global_history[$i];
      $new_url = &munge1($new_url);
      if ($glr_mode == 1) {
        $glr_index = $i;
        # %%% have to convert to a logfile index
      } elsif ($glr_mode == 2) {
        $glr_index = $i;
        if ($glr_index > $ghp) {
          $glr_index = $ghp;
        } elsif ($glr_index < 0) {
          $glr_index = 0;
        }
      } else {
        $addstack = 1;
      }
    } else {
      print "There is no entry $i in global history. Type 'hi' for list.\n";
      $skip_to_input = 1;
    }
  } elsif ($cmd eq "ghr") { #---------------------------------------------- ghr
    $glr_mode = 2;
    print "Global History Replay mode: $g_bold"."f$g_normal and $g_bold"."b$g_normal commands will now go through\nglobal history, without appending to global history or log.\nCancel by following a link or with $g_bold"."g$g_normal, $g_bold"."ho$g_normal, etc. commands.\n";
    # Start at end of global history.
    # %%% actually, if we were in glr more we should compare the GLR position
    #   to a GHR position, rather than always jumping to the end.
	$glr_index = $ghp;
    $skip_to_input = 1;
  } elsif ($cmd eq "glr") { #---------------------------------------------- glr
    $glr_mode = 1;
    print "Global Log Replay mode: $g_bold"."f$g_normal and $g_bold"."b$g_normal commands will now go through\nglobal log, without appending to global history or log.\nCancel by following a link or with $g_bold"."g$g_normal, $g_bold"."ho$g_normal, etc. commands.\n";
    # %%% find length of history, start at end
    # %%% actually, if we were already in a glr mode we should compare
    #   the GLR position to a GHR position, rather than always jumping
    #   to the end.
    $skip_to_input = 1;
  } elsif ($cmd =~ m|^gsc (.*)$|) { #-------------------------------------- gsc
    # search Nth-most recent page in cache containing a string
    # NOTE: don't need to set $glr_mode because append_gh
    # and log_gh are not called -- they are only called in the main
    # command loop. (But, we do set $addstack after finding the page)
    $parms = $1 . " ";
    if($parms =~ m|^ *\"([^\"]*)\" (.*)$|) { # "
      # quoted searchkey
      $key = $1; $parms = $2;
    } elsif ($parms =~ m|^ *([^ ]*) (.*)$|) {
      # unquoted searchkey
      $key = $1; $parms = $2;
    } else {
      print "Can't recognize parameters.\n";
      $skip_to_input = 1;
    }

    if (!($skip_to_input)) {
      if($parms =~ m|^ *([0-9]+) +([-0-9]+) *$|) {
        # passcount and initial GHP
        $passcount = $1; $ighp = $2;
      } elsif ($parms =~ m|^ *([0-9]+) *$|) {
        # passcount only
        $passcount = $1; $ighp = -1;
      } else {
        # keyword only
        $passcount = 0; $ighp = -1;
      }
    }

    if (!($skip_to_input)) {
      # Search
      $new_url = &search_gh($key, $passcount, $ighp);

      # set up command to use next time
      $default_command = "gsc $q$key$q 0 " . ($sgh_ghp-1);

      if ($new_url ne "") {
        # Then go to the page we found
        $new_url = &munge1($new_url);
        $addstack = 1;
      } else {
        print "Search pattern was not found.\n";
        $default_command = "";
        $skip_to_input = 1;
      }
    }

    # We don't want all the found URL to get added to the f/b stack, because
    # typically the user just hits return over and over again until finding
    # the desired page, and then is left with a whole lotta junk on the f/b
    # stack. However, the *last* page they have seen is probably significant.
    # So, we make it overwrite whatever was the current top item, *if* the
    # previous command was also a #gsc# command.
    $addstack = 1;
    if ($last_gsc) {
      if ($hist_ptr > 0) { $hist_ptr -= 1; }
    }
    $this_gsc = 1;
  } elsif ($cmd =~ m|^gsu +([^ ].*)$|) { #--------------------------------- gsu
    $pat = $1;
    for ($i=1; $i <= $ghp; $i++) {
      if ($matchcase) {
        $fnd = ($global_history[$i] =~ m/$pat/);
      } else {
        $fnd = ($global_history[$i] =~ m/$pat/i);
      }
      if ($fnd) {
        print $inverse . "gh $i$g_normal $global_history[$i]\n";
        $default_command = "gh $i";
      }
    }
    $skip_to_input = 1;
  } elsif ($cmd eq "hi") { #------------------------------------------------ hi
    $i = 20;
    if ($hist_ptr <= $i) {
      $i = $hist_ptr - 1;
    }
    for (; $i >= 1; $i--) {
      $u = $history[$hist_ptr - $i];
      print $inverse . "b $i$g_normal ";
      if (&url_host($u) eq &url_host($cur_url)) {
        print $bdbl;
      } else {
        print $cyan;
      }
      print "$u$g_normal\n";
    }
    $skip_to_input = 1;
  } elsif ($cmd eq "ho") { #------------------------------------------------ ho
    $glr_mode = 0;
    $new_url = $homepage_url;
    $addstack = 1;
  } elsif ($cmd =~ m|^i([0-9]+)$|) { #--------------------------------------- i
    # Check/uncheck item, submit form, etc.
    $inum = $1;
    $itype = $input_type[$inum];
    if ($itype eq "radio") {
      &check_radio($inum);
      $skip_to_input = 1;
    } elsif ($itype eq "checkbox") {
      $input_checked[$inum] = ($input_checked[$inum] eq " ") ? "*" : " ";
      $skip_to_input = 1;
    } elsif (($itype eq "submit") || ($itype eq "image")) {
      $glr_mode = 0;
      $iform = $input_form[$inum];
      $faction = $form_action[$iform];
      $fmethod = $form_method[$iform];
      $new_url = &apply_href($form_base[$iform], $faction, 1);
      $new_url .= "?";
      for($i = 1; $i <= $inp_num; $i++) {
        if ($input_form[$i] == $iform) {
          $t = $input_type[$i];
          $c = $input_checked[$i];
          if ( 0
             # The following controls which types of inputs get sent
             || ($t eq "hidden")
             || ($t eq "password")
             || (($t eq "radio") && ($c eq "*"))
             || ($t eq "select")
             || ($t eq "text")
             || ($i eq $inum)
          ) {
            $in = $input_name[$i];
            $iv = $input_value[$i];
            if ($in ne "") {
              $new_url .= &encode_field($in) . "=" . &encode_field($iv) . "&";
            }
          }
        }
      }
      $new_url =~ s|\&$||;
      $addstack = 1;
    } else {
      print "Don't understand command.\n";
      $skip_to_input = 1;
    }
  } elsif ($cmd =~ m|^i([0-9]+)[ .]o([0-9]+)$|) { #- - - - - - - - - - - - -  i
    # Select item option
    $inum = $1;
    $onum = $2;
    $ovalue = "$inum.$onum";
    $ovalue = $opt_value{$ovalue};
    $input_value[$inum] = $ovalue;
    $skip_to_input = 1;
  } elsif ($cmd =~ m|^i([0-9]+) (.+)$|) { #- - - - - - - - - - - - - - - - - i
    # Set item value
    $inum = $1;
    $ivalue = $2;
    $input_value[$inum] = $ivalue;
    # want to redisplay page from spool file
    &print_input($inum);
    $skip_to_input = 1;
  } elsif ($cmd =~ m|^i([0-9]+)\^([^^]*)\^([^^]*)$|) { #- - - - - - - - - -  i
    # Edit item value
    $inum = $1;
    $a = $2; $b = $3;
    $input_value[$inum] =~ s/$a/$b/;
    &print_input($inum);
    $skip_to_input = 1;
  } elsif ($cmd =~ m/^im ([0-9]+)$/) { #----------------------------------- im
    # display image
    $n = $1;
    $img_url = &apply_href($img_base[$n], $img_src[$n], 1);
    print "$img_url\n";
    &ev_image($img_url);
    $skip_to_input = 1;
  } elsif ($cmd =~ m/^imloc ([0-9]+) ([0-9]+)$/) { #-------------------- imloc
    # set image location
    $x = $1; $y = $2;
    $imloc = "-ge +$x+$y";
    $skip_to_input = 1;
  } elsif ($cmd eq "m") { #------------------------------------------------- m
    # more, which is like #rf# reformat but pipes output through #more#
    # or some other pager. (This used to be ideal for use on the console,
    # but now that the built-in pager accepts #mira# commands, it's more
    # of a toss-up.)
    $new_url = $cur_url;
    $retain_form = 1;
    $of_more = 1;
    $addstack = 0;
  } elsif ($cmd eq "mc") { #----------------------------------------------- mc
    # toggle case matching
    $matchcase = (! $matchcase);
    if ($matchcase) {
      print "Case matching turned on. Use #mc# again to turn off.\n";
    } else {
      print "Case matching turned off. Use #mc# again to turn on.\n";
    }
    $skip_to_input = 1;
  } elsif ($cmd eq "p") { #------------------------------------------------- p
    # print, email or save

    # First, locate the cached or file data and make the header
    $cpath1 = &cache_path($cur_url, 1);
    if (-e "$cpath1.gz") {
      $cpath2 = "$cpath1.gz";
    } else {
      $cpath2 = $cpath1;
    }
    if (-e $cpath2) {
      ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size1,
                    $atime,$mtime,$ctime,$blksize,$blocks)
                        = stat($cpath2);
      $hdr1 = "\n<!-- " . ("=-" x 33) . "- -->\n"
            . "<!-- $load_url -->\n"
            . "<!-- Cache-Date: " . localtime($mtime) . " -->\n";
      open(HDRTEMP, "> $tmpdir/hdr1");
      print HDRTEMP $hdr1;
      close HDRTEMP;

      print "$magenta(E)mail, (P)rint, (S)ave HTML, Save (A)SCII or (R)HTF, or (C)ancel:";
      $choice = <>; chop $choice;
      $choice =~ tr/A-Z/a-z/;
    } else {
        print "Sorry -- can't locate data. Did the current page load successfully?\n";
        $choice = "c";
    }

    if (($choice eq "e") || ($choice eq "m")) { # - - - - - - - - - - - -  e,m
      $cmd = &cache_path($cur_url, 1);
      # used to be in $tmpdir/spool
      &print_email(
        "mrobu",
        ($page_title eq "") ? "no title" : $page_title,
        $cmd
      );
    } elsif ($choice eq "p") { # - - - - - - - - - - - - - - - - - - - - - - p
      # Print is essentially the same as save ASCII
      $cmd = &cache_path($cur_url, 1);
      if ((-e $cmd) || (-e "$cmd.gz")) {
        $cmd = "print -2 -c2 -x cat $tmpdir/hdr1 $litname";
        print $bdmag . "Running: $magenta$cmd$g_normal\n";
        print "$cmd$g_normal\n";
        system $cmd;
      } else {
        print "Sorry -- can't locate data for $cur_url. Did the current page load successfully?\n";
      }
    } elsif ($choice eq "s") { # - - - - - - - - - - - - - - - - - - - - - - s
      print "Save (or append) to: $savetodir/";
      $path = <>; chomp $path;
      $path = "$savetodir/$path";

      if (-d $path) {
        print "That is a directory -- command cancelled.\n";
      } elsif (-e $path) {
        if ($cpath2 eq $cpath1) {
          # append, uncompressed
          $cmd = "cat $tmpdir/hdr1 $cpath2 >> '$path'";
        } else {
          # append, compressed
          system("cat $tmpdir/hdr1 >> '$path'");
          $cmd = "cat $cpath2 | zcat >> '$path'";
        }
        print $bdmag . "Appending: $magenta";
      } else {
        if ($cpath2 eq $cpath1) {
          # create, uncompressed
          $cmd = "cat $tmpdir/hdr1 $cpath2 > '$path'";
        } else {
          # create, compressed
          system("cp $tmpdir/hdr1 '$path'");
          $cmd = "cat $cpath2 | zcat >> '$path'";
        }
        print $bdmag . "Saving: $magenta";
      }
      print "$cmd$g_normal\n";
      system $cmd;
    } elsif (($choice eq "a") || ($choice eq "r")) { # - - - - - - - - - - a,r
      $cmd = &cache_path($cur_url, 1);
      if ((-e $cmd) || (-e "$cmd.gz")) {
        print "Save as: $savetodir/";
        $path = <>; chop $path;
        $path = "$savetodir/$path";

        if ($choice eq "a") {
          $args = "$tmpdir/hdr1 $litname";
        } else {
          $args = "$rhtfname";
        }

        if (-d $path) {
          print "That is a directory -- command cancelled.\n";
        } elsif (-e $path) {
          $cmd = "cat $args >> '$path'";
          print $bdmag . "Appending: $magenta";
        } else {
          $cmd = "cat $args > '$path'";
          print $bdmag . "Saving: $magenta";
        }
        print "$cmd$g_normal\n";
        system $cmd;
      } else {
        print "Sorry -- can't locate data for $cur_url. Did the current page load successfully?\n";
      }
    } else {
      print "Command cancelled.\n"
    }
    print $g_normal;
    $skip_to_input = 1;
  } elsif ($cmd eq "q") { #------------------------------------------------- q
    # quit
    $going = 0;
  } elsif ($cmd eq "r") { #------------------------------------------------- r
    # reload page

    $screen_size = " " . `stty size`;
    $ss_changed = ($screen_size ne $old_screen_size);
    $old_screen_size = $screen_size;

    $new_url = $cur_url;
    if ($ss_changed) {
      # this is a simple reformat.
      $proxy_reload = 0;
      $retain_form = 1;
    } else {
      # force reload
      $proxy_reload = 1;
    }
    $addstack = 0;
    $default_command = "r";
  } elsif ($cmd eq "rf") { #----------------------------------------------- rf
    # refresh/reformat (redisplay page, never force reload unless it wasn't
    # successfully loaded into cache)
    $new_url = $cur_url;
    $retain_form = 1;
    $proxy_reload = 0;
    $addstack = 0;
    $default_command = "rf";
  } elsif ($cmd eq "ro") { #----------------------------------------------- ro
    # reload original
    $new_url = ($orig_url eq "") ? $cur_url : $orig_url;
    $proxy_reload = 1;
    $addstack = 0;
    $default_command = "ro";
  } elsif ($cmd eq "rp") { #----------------------------------------------- rp
    # register page -- add to list of pages to monitor for changes
    open (REGFILE, ">> bk.reg");
    print REGFILE "$cur_url\n";
    close REGFILE;
    $skip_to_input = 1;
  } elsif ($cmd eq "rr") { #----------------------------------------------- rr
    # really reload
    $new_url = $cur_url;
    $proxy_reload = 1;
    $addstack = 0;
    $default_command = "rr";
  } elsif ($cmd eq "sa") { #----------------------------------------------- sa
    # show anchors
    print "$anc_num anchors:\n";
    for ($i=1; $i<=$anc_num;$i++) {
      print $inverse . "$i$g_normal ";
      print "$anchors[$i] ";
      $url1 = &apply_href($anc_base[$i],$anchors[$i], 1);
      # %%% want to use &acolor($url1, $cur_url); but munge1 call makes it different
      if (&url_visited($url1)) {
        print $g_visited_colour;
      } else {
        if (&url_host($url1) eq &url_host($cur_url)) {
          print $bdbl;
        } else {
          print $cyan;
        }
      }
      print $url1;
      print "$g_normal\n";
    }
    $skip_to_input = 1;
  } elsif ($cmd =~ m|sa +(.+)$|) { #- - - - - - - - - - - - - - - - - - - - sa
    # show anchors matching a pattern
    $npat = $1;
    print "Anchors matching $npat:\n";
    for ($i=1; $i<=$anc_num;$i++) {
      $atxt = $anc_text[$i];
      $atxt =~ s/[\001-\040]+/ /g;
      if ($atxt =~ m|$npat|i) {
        $url1 = &apply_href($anc_base[$i],$anchors[$i], 1);

        # select color for label
        # %%% want to do $acol = &acolor(&munge1($url1), $cur_url);
        if (&url_visited($url1)) {
          $acol = $g_visited_colour;
        } else {
          if (&url_host($url1) eq &url_host($cur_url)) {
            $acol = $bdbl;
          } else {
            $acol = $cyan;
          }
        }

        # print it all out
        print $inverse . "$i$g_normal ";
        print "$acol$atxt$g_normal ";
        print "$g_normal$anchors[$i] ";
        # print "$acol$url1";
        print "$g_normal\n";
      }
    }
    print "$g_normal\n";
    $skip_to_input = 1;
  } elsif ($cmd eq "sf") { #----------------------------------------------- sf
    # show frame URLs
    print "$num_frames frames:\n";
    for ($i=1; $i<=$num_frames;$i++) {
      print $frame_url[$i] . "\n";
    }
    $skip_to_input = 1;
  } elsif ($cmd eq "sg") { #----------------------------------------------- sg
    # show graphics
    print "$img_num images:\n";
    for ($i=1; $i<=$img_num;$i++) {
      print $inverse . "im $i$g_normal $img_src[$i]\n";
    }
    if ($embed_num) {
      print "$embed_num plugin items:\n";
      for ($i=1; $i<=$embed_num;$i++) {
        print $inverse . "im $i$g_normal $embed_src[$i]\n";
      }
    }
    $skip_to_input = 1;
  } elsif ($cmd eq "si") { #----------------------------------------------- si
    # show form items
    print "$inp_num items:\n";
    for ($i=1; $i<=$inp_num;$i++) {
      &print_input($i);
    }
    $skip_to_input = 1;
  } elsif ($cmd eq "tc") { #------------------------------------------------ tc
    # toggle cache load precedence
    if ($prefer_live) {
      $prefer_live = 0;
      print "MIRA will now load the cache copy before trying to go out on the net.\n";
    } else {
      $prefer_live = 1;
      print "MIRA will now go out on the net before trying to load the cache copy.\n";
    }
    $skip_to_input = 1;
  } elsif ($cmd eq "vs") { #------------------------------------------------ vs
    # view source
    $new_url = $cur_url;
    $retain_form = 1;
    $view_source = 2;
    $addstack = 0;
  } elsif ($cmd =~ m|^vs ([0-9]+)$|) { #- - - - - - - - - - - - - - - - - -  vs
    # view source at parsing level given by argument
    $new_url = $cur_url;
    $retain_form = 1;
    $view_source = $1;
    $addstack = 0;
  } elsif ($cmd eq "vse") { #---------------------------------------------- vse
    # view source in external text editor
    &ev_text($cur_url);
    $skip_to_input = 1;
  } elsif ($cmd eq ".alx") { #-------------------------------------------- .alx
    # try to load URL though Alexa
    $glr_mode = 0;
    $new_url = $cur_url;
    $new_url = "http://archive.alexa.com/cgi-bin/fetch_url?url=" . $new_url;
    $new_url .= "&user=77865415";
    $addstack = 1;
  } elsif ($cmd eq ".gpl") { #-------------------------------------------- .gpl
    # view GNU General Public License
    $glr_mode = 0;
    $new_url = $k_copyleft_url;
    $addstack = 1;
  } elsif ($cmd eq ".sci") { #-------------------------------------------- .sci
    # Show cache info
    print "   Path: $yellow" . &cache_path($cur_url, 0) . "$g_normal\n";
    $skip_to_input = 1;
#  } elsif ("$cmd 1" =~ m/^\.rlc +(\d+) /) {
  } elsif ("$cmd " =~ m/^\.rlc /) { #------------------------------------- .rlc
    # attempt reload of every page in cache
    # NOTE: don't need to set $glr_mode because append_gh
    # and log_gh are not called -- they are only called in the main
    # command loop.

    # This parameter doesn't seem to be necessary
#    $override_cache = $1;

    # save our current URL
    $temp_url = $cur_url;

    # force More to always spool all
	$defer_cmds .= " ";

    # Reload every page we've ever visited!
    &reload_gh();

    # Then reload the page we were on.
    $new_url = $temp_url;
#    $proxy_reload = $override_cache;
    $addstack = 0;
  } elsif ($cmd =~ m|^\^([^^]+)\^([^^]*)$|) { #----------------------------- ^^
    # edit current URL
    $glr_mode = 0;
    $new_url = $cur_url;
    $a = $1; $b = $2;
    $new_url =~ s/$a/$b/;
    $addstack = 1;
  } elsif ($cmd =~ m|^[0-9]+$|) { #----------------------------------------- 27
    # jump to an anchor
    if (($cmd > 0) && ($cmd <= $anc_num)) {
      $glr_mode = 0;
      $new_url = &apply_href($anc_base[$cmd], $anchors[$cmd], 1);
      $new_url = &munge1($new_url);
      $addstack = 1;
    } else {
      print "No anchor $cmd. Type $g_bold"."sa$g_normal to see a list of valid anchors.\n";
      $skip_to_input = 1;
    }
  } elsif ($cmd =~ m/^[h\?]/) { #-------------------------------------------- ?
    # help
    $y = $yellow; $n = $g_normal; $b = $g_bold; $i = $inverse;
    $g = $green; $c = $cyan; $bl = $blue;
    $t = $y;
    if ($cmd =~ m/^[h\?] +([a-z]+)$/) {
      $arg = $1;
      if ("commands" =~ m/$arg/) {
        print "               MIRA commands:
$t a $b"."href$n  go to relative URL
$t b $b#$n  back # pages (default 1) ('hi' displays stack)
$t bh$n  go back to previous host
$t bm label$n  add bookmark (with optional label)
$t dl $b#$n  download target of link as binary data
$t dl im $b#$n  download image as binary data
$t dl $b\"href\"$n  download target of relative URL as binary data
$t ev$n  view current page through external viewer $b$eviewer_html$n
$t evt$n  view current page through external viewer $b$eviewer_txt$n
$t eva$n  view current page in ASCII through external viewer $b$eviewer_src$n
$t f$n  forward
$t g $b"."url$n  go to absolute URL
$t g $c"."web1.abc.org$n  go to $c"."http://web1.abc.org$n
$t g $c"."abc$n  go to $c"."http://www.abc.com$n
$t ga $c"."Abc$n  go to anchor whose label contains $c"."Abc$n
$t gb$n  visit the bookmarks page $c$bm_url$n
$t gb $b"."abc$n  go to recentmost bookmark whose name includes $b"."abc$n
$t gh$n  show global history (latest 20 URLs ever visited)
$t gh $b#$n  go to page from global history
$t ghr$n  enter Global History Review mode -- $b"."f$n, $b"."b$n and $b"."gh$n commands
        review Global history without modifying it.
$t gsc $b"."foo N$n  search global history for Nth most recent page that
        contains $b"."foo$n
$t gsu $b"."foo$n  show entries in global history that contain $b"."foo$n in
        the URL.
$t h$n  help
$t hi$n  history (show top #n# URLs on stack)
$t ho$n  go to home page $c$homepage_url$n
$t i$b# text$n  set contents of form item
$t i$b#$t^$b"."texta$t^$b"."textb$n  edit text in form item
$t i$b# $t"."o$b#$n  select item option
$t i$b#$n  check/uncheck a checkbox, submit form, etc.
$t im $b#$n  display image with external viewer $b$eviewer_img$n
$t m$n  view current page $screen_height lines at a time
$t mc$n  match-case toggle for $b"."gsc$n and $b"."gsu$n commands
$t p$n  print, email, or save the current page
$t q$n  quit
$t r$n  reload current page, (or just refresh display if you changed the
     window size)
$t rf$n  refresh current page, (reformat without reloading from the Internet)
$t ro$n  reload original URL (before redirect). This is useful when $t"."r$n
     doesn't work because the original is cached and redirects to another
     cached URL.
$t rr$n  really reload current page (same as $t"."r$n unless you just changed
     the window size)
$t sa$n  show list of anchors and their HREFs
$t sf$n  show list of frames and their URLs
$t sg$n  show list of graphic (inline image) items
$t si$n  show list of input items
$t vs $b#$n  view source, supply filter level from 1 to 10 or omit to see raw source
$t vse$n  view source in external viewer $b$eviewer_vse$n
$t ^$b"."texta$t^$b"."textb$n  edit text in current URL and load resulting URL
$t $b#$n  go to numbered link from current page

$t imloc $b# #$n set coordinates of the windows images are displayed in

Multiple commands can be separated by $t;$n characters.
";
        $default_command = "h intro";
      } elsif ("intro" =~ m/$arg/) {
        print "               Introduction to MIRA

$cmdhelp

- Distributed in source code form under the GNU General Public License

- Allow for user customization and third-party extension through a
  preferences file that can include source code for plug-ins, patches
  and new features

- Provide a powerful command-line interface (with such things as
  macros, anchorname completion, global history completion, and lots
  of other shortcuts) to make the browser as easy to use as possible.

Not all of these objectives have been met and MIRA is currently in an
alpha phase.  If you want to be involved in the beta test program, and
are running on a UNIX(tm)-compatible system with Perl and an
Internet connection, contact the author $bl"."mrob at mrob com$n.
";
        $default_command = "h tutorial";
      } elsif ("tutorial" =~ m/$arg/) {
        print "               MIRA tutorial

Links on web pages, and the output of certain commands like $t"."hi$n contain
bits of text in $i"."inverse$n. Anything that is displayed in inverse
is a command that can be typed. These can be any of the following:

  $i"."27$n A bare number is an anchor. Type $i"."27$n to go to another page.
  $i"."im 3$n An image can be viewed with the $i"."im$n command.
  $i"."i3$n An item on a form that can be filled in (like a text entry box) is
      labeled like this. To enter the text 'foo', you would type $i"."i3 foo$n.
  $i"."i3.o4$n When an item in a form has options, each option is chosen
      with a command like $i"."i3.o4$n.
  $i"."b 3$n The $i"."hi$n command, which gives the current history (pages
      you've just visited) gives a list where each item starts with one of
      these commands. You could type $i"."b 3$n to go back 3, which would 
      go to the indicated URL.
  $i"."gh 3$n The $i"."gh$n command, which gives global history (all pages
      you've ever visited) gives a list where each item starts with one of
      these commands. You could type $i"."gh 83$n to go to the 83rd entry
      in global history, which would go to the indicated URL.

Links are displayed in three different colors:
$bl blue$n  - unvisited link on the same site as the current page
$c cyan$n  - unvisited link on a different site
$g green$n - visited link
";
        $default_command = "h stack";
      } elsif ("stack" =~ m/$arg/) {
        print "               About the Stack, History and Log

Stack: The \"stack\" contains the URLs that you have recently visited,
which you can return to with the $i"."b$n and $i"."f$n commands, and which is
displayed by the $i"."hi$n command.

The stack works just like the \"forward\" and \"back\" buttons in graphical
browsers. When you type $i"."b$n, you go back to where you just were, and
then if you want you can go \"forward\" again with $i"."f$n, but if you go
to a new URL instead the place you would have gone \"forward\" to is
forgotten. It is called a \"stack\" because the URLs are like looking at
a stack of papers, where you can only see the top one, and when you
go back to earlier ones, you get them in reverse order from how you
originally saw them.

Global history: This is a list of URLs that records every place you've
been to. URLs stay in the global history even when you type $i"."b$n and
go to a new URL -- the new URL just gets added to the end. In fact,
*every* time you go to a URL it gets put on the end of the list, but
if you've been there before, the URL gets removed from its original
place in the list -- so each different URL only appears once in the
list no matter how many times you go to it.

You can view the global history with $i"."gh$n and search it with $i"."gsu$n.

Global log: This is like the global history, except that URLs never get
deleted from the middle of the list when you revisit them. So, if you've
been to $c"."www.acme.com$n 27 times, $c"."http://www.acme.com/$n will appear in the
global log 27 times.

$b"."mira$n saves the stack, global history and global log in files, which
are updated every time you enter a command. You can quit $b"."mira$n any
time you want and restart later, and all three will be just the way
you left them. You will find this particularly useful if you often
leave \"unfinished business\" on the stack and rely on it as a reminder of
things you intend to do.
";
        $default_command = "h multiple";
      } elsif ("multiple" =~ m/$arg/) {
        print "
Multiple simultaneous $b"."mira$n sessions

(read $i"."h stack$n first if you are unfamiliar with the stack, global
history and global log)

The stack is updated to disk every time you go to a new URL, which
happens with most commands including $i"."b$n and $i"."f$n. When you start
a new $b"."mira$n session, it gets the latest stack that was saved. This
means that if you run multiple $b"."mira$n"."s, the last one that went to
a new URL will be the one whose stack is saved, regardless of what
order you quit them in. To control which stack gets saved, first
quit all the $b"."mira$n"."s you don't want saved, then in the last $b"."mira$n
type:

   $i"."b;f;q$n

The first two commands $i"."b;f$n cause $b"."mira$n to switch URLs, which causes the
stack to get saved.

Global history: Like the stack, each $b"."mira$n keeps its own copy in
memory. But, every time you visit a new URL, the URL is added to the
global history file located in:

   $miradir/g_hist

and the contents of that file are used to construct the memory copy of
the global history when a new $b"."mira$n is started. There are two
consequences of this:

  - If you run multiple $b"."mira$n"."s and are visiting URLs with both of
    them, the URLs get added to the history file in whatever order you
    do the visiting, so if you're switching back and forth between the
    different $b"."mira$n"."s, the order of the URLs in the file will
    reflect that.

  - This interleaving of URLs from two or more $b"."mira$n"."s won't show
    up in the $i"."gh$n command until you launch a new $b"."mira$n.
";
        $default_command = "h legal";
      } elsif ("legal" =~ m/$arg/) {
        print "
Legal issues

MIRA is copyright (c) 1998-2012 by Robert P. Munafo.

All use of MIRA is subject to the terms and conditions of the GNU
General Public License. For details, type $i.gpl$n or visit
$c$k_copyleft_url$n.

Verbatim copying and distribution is permitted in any medium, provided
this notice is preserved and the terms of the GPL are followed.

";
      } else {
        print "Don't have help for $i$arg$n. Type $i"."h$n for help.\n";
        $default_command = "h";
      }
    } else {
    print "MIRA - command-line interface, text-only, web browser

Help commands (these can be abbreviated):

 $i" . "h intro$n for an introduction to $b"."mira$n.
 $i" . "h commands$n for a list of commands.
 $i" . "h tutorial$n for a tutorial.
 $i" . "h stack$n for a description of the stack, history and log.
 $i" . "h multiple$n for a description of issues related to running more
          than one $b"."mira$n at once.
 $i" . "h legal$n for copying, distribution and other legal issues.

You can see all of these help pages by just hitting <return> a few times...
";
    $default_command = "h commands";
    }
    $skip_to_input = 1;
  } else {
    print "Didn't understand command '$cmd'. Type h for help.\n";
    $skip_to_input = 1;
  }

  if (!($skip_to_input)) {
    # Here's where we handle any jumps to non-html URL's
    #
    if ($going && (&is_graphic($new_url))) {
      &ev_image($new_url);
      $skip_to_input = 1;
    } elsif ($going && (&is_pdf($new_url))) {
      &ev_pdf($new_url);
      $skip_to_input = 1;
    } elsif ($going && (&is_audio($new_url))) {
      &ev_audio($new_url);
      $skip_to_input = 1;
    } elsif ($going && (&is_email($new_url))) {
      &do_email($new_url);
      $skip_to_input = 1;
    }

    $cur_url = $new_url;
  }
}

# clean up!
if ($htps_path ne "") {
  unlink($htps_path);
}

# &export_gh();

# ideas for commands:
# macros -- name starts with '.'
#
# paging ideas:
#  <return> to show next page (prompt at proc09 level and add "abort load" variable to communicate down to main loop)
#  <return> on end of page restarts at top of same page (requires reloading from file)
#
# bh - go back to previous host (i.e. repeat the #b# command until reaching a
#    URL that is on a different host; useful for checking the next link in
#    a HotBot search after you're "done looking" at a site)
# dm - define macro
# glr - enter global log replay mode: while in GLR mode, #f# and #b# commands browse global log file (which records pages in the exact order they were visited, not the same as the "gh" history) and do not add to the "gh" global history or "hi" stack. GLR mode is cancelled by any command in the set: {a, g, gb, ho, <^a^b>, <#>}
#    (implementation: never call append_gh, current position is glr_index; glr_mode is T iff in GLR mode)
# glr - if repeated, turns off GLR mode and jumps back to end of global history
# glb - same as ghr, b
# glf - same as ghr, f
# gh - in GHR mode or GLR mode, should print neighborhood of current position
#      with corresponding #f# and #b# commands
# ghr, ghb, ghf - like glr, glb, glf but operate on the global history array, which is ordered differently than the global log
# gl - list URLs in global history. differs from gh in two ways: it lists the log, including all duplicates, and it displays the results through the browser by creating HTML in a temp file and visiting it, so you can use links to access pages, scroll, etc. Also numbers each entry for use in the #vle# command.
# rc - check all registered pages to see if they have changed or disappeared (if changed, old *and* new versions are kept in permenent cache)
# rcr - redisplay list of pages found by last rc command (to facilitate keeping track of which ones you haven't yet revisited)
# rp - "register" page (and store in permanent cache)
# s - save current file (in source form)
# vle - set visited-links-expiration pointer to an index from the listing created by #gl#.
#
# would like to add commands to #ping#, #traceroute#, etc. to the current URL's host
#
# Changes to make:
# vse - should view source for all frames (in different windows of the Emacs session)
# .rlc #pattern# - reloads only those URL's that match a pattern (like eww6n)
#                - rlc should also load *only* those pages that aren't in the
#                  cache
# <return> to go to auto-refresh URL unless there is already a more important default command
#
# some mechanism for detecting duplicate files in the cache and replacing duplicates with a linkfile.
#
# bugs:
# formatting anomalies in HotBot results happen because HotBot sends different HTML depending on what browser you're running.


#
# The place to announce MIRA is comp.infosystems.www.browsers.misc

# Want to make it save the history stack and restore on launch.