Formq.cgi Demos and Documentation

Formq.cgi Documentation

Introduction

Formq.cgi is a small Perl program designed to allow quick form development. The program processes simple HTML forms under user control. Its default operation can be modified by including various hidden type fields in the form, and it is easy to customize this default behavior.

It is written as a single program file. It handles repeat field names, but only handles single part forms. It was developed with Perl 5.003 on a Digital Unix V4.0B, and tested with Navig 4.04 & 4.05 (WinNT 4.0); it has also been tested with Navig 3.0 Gold (Win3.1), Navig 3.01 (WinNT 4.0), ...

This documentation applies to the NEIU web server environment. You will have to adapt these instructions for other server environments.

Installation

Copy the formq.cgi file into your http directory or any http subdirectory. For ease of both absolute and relative addressing it is recommended that you place formq.cgi in your http directory.

Copy formq.cgi to your account with one of these methods:

1. Login to your NEIU Unix account and give this command (you must be in the C or Korn shell):

cp -p ~rspatema/http/formq.cgi ~/http

If formq.cgi already exists in your http directory, the first command will prompt you with an overwrite warning, to which you should give 'y', assuming you want to update your copy.

2. View source, select menu item 'File', choose 'Save As ...', rename to formq.cgi, click 'Save', ftp to your http directory in your account, give the Unix command 'chmod 755 ~/http/formq.cgi'.
DO NOT right click on the View Source link and use 'Save Link As...' or Shift-Click on View Source and 'Save'; these options have been disabled because they save extra characters that are used to correctly display the Perl file.
View Source

Demos

These demos illustrate various features of formq.cgi and show some of its typical uses. You can select and copy all or part of the HTML files to help in your form creation efforts. DO NOT right click on the View Source links and use 'Save Link As...'; this option has been disabled since it saves extra characters that are used to correctly display the HTML files.

Demo	View	Description

Use

1. Setup the form and set the form's action attribute to formq.cgi with an absolute or relative reference. For example:

<form action="http://www.neiu.edu/~username/formq.cgi">
<form action="../formq.cgi">

The first line is an absolute reference which assumes that formq.cgi is in the http directory of username, whereas the second line is a relative reference which assumes that the HTML document is in a directory immediately below the http directory, where formq.cgi resides.

2. Add the two or three required hidden fields to the form. For example:

<input type="hidden" name="date_file"value="reg/user.data"> <input type="hidden" name="emailto" value="username@neiu.edu"> <input type="hidden" name="vars" value="name,address,city,st,zip">

asks for both data file storage (in the user.data file in the reg subdirectory of http) and email notification (sent to username at neiu.edu) and specifies the form fields to be saved and emailed, and the order in which they'll appear in both. Either 'data_file' or 'emailto' must be given, and both are allowed. Mention file permissions.

3. Add any optional hidden fields to tailor the form behavior as needed. See below for a full list of hidden field names and their default values. For example:
<input type="hidden" name="req_vars" value="name"> <input type="hidden" name="response" value="Your name and address have been recorded."> <input type="hidden" name="return_url" value="index.html"> These fields specify 1) that only the 'name' field must be non-blank for the form data to be accepted, 2) the response to valid input will be the value shown, and 3) the return link, after valid input, will be to the 'index.html' file in the http directory.

3. If the 'data_file' hidden field is used, create the data file by loging in to your Unix account, changing to the target directory, and giving this command:

cp -p /dev/null filename.data

replacing 'filename.data' with your specific filename. You can also create the data file with a Unix editor such as pico, vi, etc, but you must then give the Unix command: 'chmod 666 filename.data'.

Form Design and Form Element Notes

1. Unique field names. Data encoding, self describing data vs $fileFieldNames=1, for field types Select, Check Box, Radio Button

2. If Radio Button or Check Box elements are used the 'allow_check_button' field should be set to 1. This turns off field name testing during form processing and requires the form designer to carefully check the field names given in the 'vars' field.

3. Repeating field names are handled by joining the data of all fields with the same name and separating them from each other with the 'repeat_field_delim'. Repeating field names can be used with fields of any type, but are most useful with Check Box type fields, where the user is allowed to select one or more responses to a question. See Demo-2 for an example.

4. Multiple lines, including new line characters in Textarea fields are handled in the following manner:
o In the HTML response, new line characters in text cause a line break.
o Text containing new line characters cause a line break in the email field, with a small indent when field names are used.
o Since new line characters normally mark the end of each record in the data file, new line characters in text are converted to 'new_line_delim' values.

Email Notes

1. The 'emailto' address is tested for validity when the email is sent. If it is invalid, the sendmail program relays an error message to the top of the HTML response page, above the normal 'response' message.

2. The fields 'emailfrom' and 'email_subject' are often assigned to user input text fields; if used this way, do not include hidden fields of the same name. If these fields have non-blank values they are included in the email header when the email is sent, and they don't need to be in the 'vars' list for this to occur.

3. The 'emailfrom' field, whether included as a hidden field or user input text field, is NOT tested for a valid address when the email is sent. Invalid addresses will only be detected when this field is later used as an email destination address.

4. For security reasons, NEVER make the 'emailto' or 'emailcc' fields user supplied text input. Malicious emails could be sent to any address if this is done. If the user needs to select one of several email addresses, for example to send a comment to any one of a department's professors, use a Select element, which will produce a drop down list for the user's choice. See Demo-x.

Data File Notes

1. When a form is successfully processed a new record is appended to the data file, if used. The data fields are in the sequence given in the 'vars' field, each separated by 'file_field_delim' values (default is tab), and the record is terminated with 'file_record_delim' value (default is a new line character). The file is locked during writing and then unlocked, to prevent simultaneous access overwrite problems.

2. Using the default 'file_field_delim' value (a tab character) and the default 'file_record_delim' value (a new line character) allows the file to be downloaded to a local computer, then imported to a spreadsheet program like Microsoft's Excel. How to open file, sort by field, etc.

Hidden Field Table

This table lists the form hidden field names and corresponding Perl program variable names, along with their default values. If a hidden field is not included in the form, the default value will be used in form processing. It is easy to create custom formq.cgi's with different default values; see the Customization Notes. Field Names in bold are the ones most commonly used in forms.

Form Hidden Field Name (case sensitive)	Default Value (Perl Syntax)	Perl Program Variable Name (case sensitive)	Description
			Destination and Field Control.
data_file	-----	$dataFile	Required - 'data_file' or 'emailto', and both can be given. If 'data_file' is given a record containing the 'vars' field values is appended to it. If 'emailto' is given an email containing the 'vars' field values is sent to this address.
emailto	-----	$emailTo
vars	-----	@var	Required. Comma separated list of form fields to write to the data file (in sequence given) and to enclose in the email. These can be user input fields and/or hidden fields.
req_vars	all vars	@reqVar	Optional. Comma separated list of form fields that must be non-blank for form data to be accepted. If not given, all fields in 'vars' are required. These can be given in any order.
allow_check_ button	0	$allowCheckButton	Optional. 0=no, 1=yes. Allows the use of Check Boxes and Radio Buttons in the form. This also disables 'vars' field name checking, so the form builder must check these names carefully.
repeat_field_ delim	" " (space)	$repeatFieldDelim	Optional. The delimitor between the data values of fields with the same name, eg Check Boxes.
			Email Control. All Optional.
emailfrom	"" (blank)	$emailFrom	The value of 'emailfrom' is the address of sender, which is placed in the 'From' field of the email header. The value of 'email_subject' is placed in the 'Subject' field of the email header. These fields are often text fields for user input, not 'Hidden' fields.
email_subject	"Form Submission"	$emailSubject
emailcc	"" (blank)	$emailCC
email_field_ delim	"\n" (new line)	$emailFieldDelim
email_time_ stamp	1	$emailTimeStamp	0=no, 1=yes.
email_field_ names	1	$emailFieldNames	0=off, 1=on.
-----	"/usr/lib/ sendmail"	$emailProgram	Name of email processing program on the server. Usually does not have to be changed.
			Data File Control. All Optional.
file_field_delim	"\t" (tab)	$fileFieldDelim	Delimiter separating field values in the data file.
file_record_ delim	"\n" (new line)	$fileRecordDelim
new_line_delim	" "	$newLineDelim
file_time_ stamp	1	$fileTimeStamp	0=off, 1=on. Place a time & date stamp at the end of each data file record.
file_field_ names	0	$fileFieldNames	0=off, 1=on.
trim_data	1	$trimData	0=off, 1=on. Remove leading and trailing spaces from each data value before saving to the data file.
			Response Control. All Optional.
-----	"Form Results"	$respTitle	Title of HTML response page.
response	"Your form has been processed. Thank you."	$respMsg	Single line of text centered at the top of the response page after an error-free form submit.
return_url	"" (blank)	$returnURL	The return url address for normal form processing. If a relative address is used, it is relative to the location of formq.cgi on the server. If smart return is off, this address is also used for forms containing errors.
return_name	"Return"	$returnName	Name of the return url link, shown on the response page.
smart_return	1	$smartReturn	0=off, 1=on. Sets the return link to point to the previous form and its data when required, but blank fields (or other errors) are detected.
smart_return_ name	"Return to Active Form"	$smartReturnName	Name of the return url link when an incomplete form is submitted.
show_form_ data	1	$showFormData	0=off, 1=on. Display all the form's field names and their corresponding values on the response page after an error-free form submit.
bg_color	"FFFFFF" (=white)	$bgColor	Response page background color and text color. These are given in hexidecimal RGB format, eg "FF0000"=red, "FF00FF"=violet, etc.
text_color	"000000" (=black)	$textColor
			Security Control.
-----	1	$security	0=off, 1=on. Security tests the referring page's domain against the valid domain list, disallowing any non-valid references.
-----	"orion.neiu.edu", "gamut.neiu.edu", "www.neiu.edu"	$validDomains	Comma separated list of quoted valid domain names. If the list is empty, security will check the referring page's domain against the 'SERVER_NAME' environmental variable. Case is unimportant since testing is done with case insensitivity.

HTML-Form	Name	Perl Program	HTML-Form	Name	Perl Program
Special Character Table
Character strings appear the same in both HTML and Perl except for the following special characters. If you use any of these in the HTML or Perl program you MUST code them as shown. Two or more of these characters can be grouped together. Examples:
" "	space	" "	"<"	less then	"<"
" "	tab	"\t"	">"	greater then	">"
			"&"	ampersand	"\&"
			"""	quote	"\""

Relative vs. Absolute Addressing (Review)
At three points in setting up a form using formq.cgi an address may be required:
a) b) c)	the action attribute of the form tag (an internet address). the 'data_file' value, if used (a server file address). the 'return_url value, if used (an internet address).
Absolute addressing can be used for all three but it is easier to use relative addressing, especially if the form will be moved or copied between accounts on the server. In the following examples assume we have the following directories, and files located in them, and that eval.html is the file containing the form:
	`username/http/course/survey \| \| \| formq.cgi \| eval.html & eval.data index.html`
Absolute (use 'staff' or 'student' in 'data_file' value):
	`<form action="http://www.neiu.edu/~username/formq.cgi"> <input type="hidden" name="data_file" value="/home/staff/username/http/course/survey/eval.data"> <input type="hidden" name="return_url" value="http://www.neiu.edu/~username/course/index.html">`
Relative (action is relative to eval.html, the next two are relative to the formq.cgi location):
	`<form action="../../formq.cgi"> <input type="hidden" name="data_file" value="course/survey/eval.data"> <input type="hidden" name="return_url" value="course/index.html">`
Note that with relative addresing neither the server name (www.neiu.edu) nor the account name (username) is used, so transferring the form to another account is simplified, and adjustments are needed only if the directory names or directory structure are different.
Program Operation and Security Notes
1.	When security is on (Perl variable $security=1) the form's url is checked against the list of valid domains (@validDomain) and stops any form processing from an invalid domain. This prevents forms not from one of the valid domains from using the formq.cgi program in your account. This is not 100% foolproof, but it greatly impeeds malicious use. If at all possible, DO NOT TURN OFF SEURITY CHECKING.
2.	The formq.cgi program can handle both GET type submits (for small forms only) and POST type submits (for any size form).
3.	Email is sent first, data file record is written second. Why.
Misc. Notes
1.	Hidden fields with other names, include in 'vars' list. See Demo-x and muscon demos.
2.	All delim fields/variables can be 0, 1, 2, ... characters in length. Ex of "\n\n" in Demo-x.
3.	Change <code> text color; split hidden field table into common fields, uncommon fields; how to update url w/o updating window; check Matt's script archive;

HTML-Form	Name	Perl Program	HTML-Form	Name	Perl Program
Special Character Table
Character strings appear the same in both HTML and Perl except for the following special characters. If you use any of these in the HTML or Perl program you MUST code them as shown. Two or more of these characters can be grouped together. Examples:
" "	space	" "	"<"	less then	"<"
" "	tab	"\t"	">"	greater then	">"
			"&"	ampersand	"\&"
			"""	quote	"\""

HTML-Form	Name	Perl Program	HTML-Form	Name	Perl Program
Special Character Table
Character strings appear the same in both HTML and Perl except for the following special characters. If you use any of these in the HTML or Perl program you MUST code them as shown. Two or more of these characters can be grouped together. Examples:
" "	space	" "	"<"	less then	"<"
" "	tab	"\t"	">"	greater then	">"
			"&"	ampersand	"\&"
			"""	quote	"\""

HTML-Form	Name	Perl Program	HTML-Form	Name	Perl Program
Special Character Table
Character strings appear the same in both HTML and Perl except for the following special characters. If you use any of these in the HTML or Perl program you MUST code them as shown. Two or more of these characters can be grouped together. Examples:
" "	space	" "	"<"	less then	"<"
" "	tab	"\t"	">"	greater then	">"
			"&"	ampersand	"\&"
			"""	quote	"\""