first2html

Paul H Alfille - 2003

• Synopsis

  first2html is a set of programs to convert PFS:First Choice database files to a web page. It is Open Source, safe, well documented, and easily modifyable. Examples are included.

  First Choice is a DOS-based flat file database. This program allows display and formatting of First Choice databases in a web browser.

  There are actually 5 programs included in this suite, as listed in the following table. first2html is the main routine, with the others either allowing better presentation, or error reporting.

  Program	Arguments	Language	Description
  first2html fc2html.exe (DOS)	First Choice database filename	C	Takes a .FOL file as input, and writes an HTML table as output. No formatting except <TABLE><TR><TH><TD> tags. One row per text line. Field contents are trimmed of leading and trailing whitespace, and special characters are HTML-escaped.
  first2tell fc2tell.exe (DOS)	First Choice database filename	C	Takes a .FOL file as input, and writes an HTML-formatted report of summary information. Report includes file length, number of active records and deleted blocks. Also includes field information: number, names, lengths. Erors in access and internal inconsistencies are also listed.
  first2list fc2list (DOS)	Optional: list of fields to include in order i.e. "1,3,7,4" quoted and comma separated. Ranges "2-7" or "3-" can be used. Combinations and repeats can be used "1,2-7,6". Field numbering starts at 1. Default: All fields, in order.	perl	Easily modified "wrapper" script. Takes first2html input and parses each line, allowing prettier web formatting and filtering. Field can be selected for presence and order. You can also add cell formatting.
  first2sort fc2sort (DOS)	Optional: [field list - display]:[field list - sort]:[repeat suppression] Field lists follow the same syntax as first2list argument. Repeat suppression is the number of fields (from left) where duplicated entries will not be listed. This mimics a hierarchial display. Default: All fields, in order : All fields, in order : All displayed fields	perl	Same as first2list (parsing the output of first2html) but also sorts on field content. Optionally suppress repeated entries in a column.
  first2web fc2web (DOS)	none	perl	Easily modified "wrapper" script. Takes first2html input (or first2tell, first2list, or first2sort) and adds HTML,HEAD,TITLE and BODY sections. Great place to put in page navigation and page colors.

• Quick Start

  1. Download the version appropriate to your platform (Windows or Linux/Unix)
  2. Decompress the file into it's own directory. Use
     Windows: pkunzip -d first2ht.zip
     Linux: tar -xz first2html.tar.gz.
  3. Use the pre-compiled binaries (at least for now).
  4. At a command prompt, in that directory, type
     first2html filename | first2web > temp.html
     where filename is the name of your First Choice database file (usually something like CONTACTS.FOL).
  5. Use a Web browser (eg. Netscape) to open the file temp.html
  6. Great! It works. Now start fiddling.
  7. Look at the example directories for ways to add the web interface for file selection.
  8. Look at the formatting, sorting and automatic generation examples.

• About First Choice

  First Choice is a DOS-based flat file database. See The Secret Guide to Computers for more historical information. First Choice allows arbitrary length text fields, decent report capabilities, and query by example (to use a Paradox term). The file format (with an FOL extension) is proprietary.

  First Choice has been called abandonware, however the current owner of the product, Broderbund, has some assitance on their web site. Old copies of the software appear to still be available.

  If you still use First Choice, or have data in this file format, you may want a way to access the data. There are several alternatives:

  1. Use First Choice's file exporting capabilities. (Needs a working version of the program, and cannot be automated).
  2. Use first2html. (Read-only access, easily automated, HTML output).
  3. Commercial products, like First Out from Advanced Computer Innovations, Inc. (I havent' tested this approach.)

• Windows

  Both First Choice and first2html should work in a DOS box under windows, or even under MSDOS. compiled programs are included, as well as the source code. To modify and compile the source code you can use the following resources:

  1. C compiler

     There shouldn't that great a call to modify the C source code, but any standard ANSI-C compiler will work. Besides the traditional offering from Microsoft, there are free choices from DJGPP or HI-TECH's Pacific C. Complete list. Pacific C was used to generate the included binaries. Project files are included. (Compiled for 8086 platform, works under DOSEMU as well).

  2. perl

     Perl is easy. Either get the official port from the CPAN archive or use the very convenient but not free perl2exe.

• Linux

  Linux is easy to set up for this project. All the tools are present, and this project was actually done on Linux. First Choice will work under linux using dosemu (FreeDOS as operating system, VGA resolution). Most linux distributions include the C compiler and perl interpreter. First Choice can also be run on a DOS/Windows machine with the files accessable to the linux box via Samba. Finally, there is the redoubtable Apache web server to power your web site.

• first2html

  first2html reads a First Choice database file and outputs an HTML table. The database program is a proprietary format that I've reverse-engineered. Files have the .FOL extension. See the current limitations on the database design.

  The First Choice filename is the sole command line argument, and is required.

  Output is a bare HTML table, with the following format (each ends with a carriage return):

  • TABLE line
  • Table headings TR TH /TH /TR
  • Table row TR TD /TD /TR
  • More table rows TR TD /TD /TR
  • End table /TABLE

  The output is intended to be sent to another one of the included programs, first2list or first2sort and finally first2web. Use first2tell initially to analyze the First Choice database.

  Under DOS or Windows {R} the executable is named FC2HTML.EXE to get around DOS file naming constraints. The program should run under any version of DOS or Windows, but has only been tested under FreeDOS, MSDOS6.2 and Win98.

• first2tell

  first2tell reads a First Choice database file and outputs an HTML file giving database analysis and summary information. Files have the .FOL extension. See the current limitations on the database design.

  The First Choice filename is the sole command line arguement, and is required.

  Output is a bare HTML file, without HTML HEAD TITLE or BODY tags. It is suitable for embedding in other HTML documents, or piping to the first2web program (included in this package).

  Under DOS or Windows {R} the executable is named FC2TELL.EXE to get around DOS file naming constraints. The program should run under any version of DOS or Windows, but has only been tested under FreeDOS, MSDOS6.2 and Win98.

• first2list

  first2list takes it's input from the first2html program and selects fields to display. Input from first2list and first2sort will also work, but probably makes little sense.

  Optional argument: "display-fields:sort-fields:norepeat"

  The argument format is actually quite intuitive, and the defaults very rational. Field lists are the same as in the first2list program.

  Optional argument: a list of fields to include, in the order specified. Enclose in quotes. Examples:

  • "1,2,3" -- first three fields
  • "1-3" -- first three fields
  • "-3,7" -- first three fields, and 7th field
  • "3,2,1" -- different order
  • "" -- blank or omitted -- all fields in order

  Output has the same format as the input, an HTML table. It can in turn feed the first2list or first2sort and finally the first2web program (included in this package).

  This program is written simply, and well documented, in PERL. It can easily be modified on a per-record basis to process fields. Possible methods include

  • entry translation or lookup
  • splitting fields apart
  • splitting records into multiple records
  • filtering out records based on field contents

  TABLE attributes are preserved, but field attributes (e.g. TD BGCOLOR=BLUE) are not. Modify the program code (should be easy) to change appearance.

  If customized processing is done, the ability to further process the output with first2list and first2sort becomes especially powerful.

• first2sort

  first2sort takes it's input from the first2html or first2list program and selects fields to display, sorted. Input from first2sort will also work, but probably make little sense.

  Some examples:

  • "1,2,3" -- (same as "1,2,3:1,2,3:3") display first three fields, sorted in that order, and suppress all repeated fields
  • "1-3:4" -- (same as "1,2,3:4:3") display first 3 fields, after sorting on fourth, and suppress all repeated fields
  • "" -- blank or omitted -- all fields in order, for display, sorting and suppression

  Output has the same format as the input, an HTML table. It can in turn feed the first2list or first2sort and finally the first2web program (included in this package).

  Display is an HTML table, with the field headers and column contents from the display-fields list.

  Sorting is done on the specified fields, in the order given. HTML escaping may complicate non-alphanumeric order. Record order should be preserved for duplicate sort fields (at least for the first 1000000 records). Sort order comes from PERLs hash key sorting function.

  Repeat suppression is a simple suppression of repeated fields. An example:

  No repeat suppression Year Region Name 1983 Burgundy Gevrey-Chambertin, Clos de la Justice, Bourre 1983 Burgundy Nuits-St.-Georges, 1er Cru, M. de Gramont 1985 Burgundy Chassagne-Montrachet, 1er Cru, Jean-Noel Gagnard 1985 Burgundy Fixin, Mongeard-Mugneret

  Repeat suppression Year Region Name 1983 Burgundy Gevrey-Chambertin, Clos de la Justice, Bourre     Nuits-St.-Georges, 1er Cru, M. de Gramont 1985 Burgundy Chassagne-Montrachet, 1er Cru, Jean-Noel Gagnard     Fixin, Mongeard-Mugneret

  Repeat suppression, different order Region Year Name Burgundy 1983 Gevrey-Chambertin, Clos de la Justice, Bourre     Nuits-St.-Georges, 1er Cru, M. de Gramont   1985 Chassagne-Montrachet, 1er Cru, Jean-Noel Gagnard     Fixin, Mongeard-Mugneret

  This program is written relatively simply, and well documented, in PERL. It is probably easier to do per-record processing in first2list and pipe that output into first2sort since the sorting adds complexity. Performing summary or cumulation functions on the data should work well, however.

  TABLE attributes are preserved, but field attributes (e.g. TD BGCOLOR=BLUE) are not. Modify the program code (should be easy) to change appearance.

  Suppressed fields are (invisibly) replaced with "&nbsp" -- the HTML encoded "non-breaking space." This allows first2list and first2sort to recover the suppressed fields when given sorted input.

• first2web

  first2web surrounds it's input with the trappings of a full HTML document: Content type, HTML, BODY, etc.

  Input is the product of any of the other routines, first2html, first2tell, first2list, or first2sort.

  Output is an HTML document that can be returned from the Web server.

  This program is written very simply, and well documented, in PERL. It should be modified to change title, headings, colors, fonts, etc.

• Constraints

  Limitations of the first2x programs
  (not First Choice)

  Database	PFS:First Choice v3.5
  Number of records	Unlimited
  Number of fields	Unlimited (100 for first2tell summaries)
  Length of fields	~65000 characters
  Field types	general
  Form design	Each line must have a field, or be blank

• Security

  Good news! first2html was designed to be safe.

  The C programs: first2html and first2what take only a filename as an argument, dynamically allocate no memory, open the database file in read-only mode, and check the index for the array before each access. Even a maliciously constructed input file could only account for output about 5 times it's length. The output is an HTML table, with cell contents "web escaped."

  The perl programs first2list, first2sort, and first2web perform no shell routines. They perform i/o only on stdin and stdout. first2sort places the entire database in memory, potentially twice (the keys and the data).

  Obviously, permission for selecting and reading the databases should be carefully considered.

• License and Copyright

  Written by Paul H Alfille (palfille@partners.org), copyright Feb 2003.

  All code is public domain, specifically under the MIT license (I'm an alumnus).

  I'd be interested in feedback on the quality, documentation, or usefulness of first2html. Drop me a line!

• Software Download

  • first2html.1.tar.gz