Data Forms

Structured data, how to set up and work with data forms.

Overview

Next to freeform topic contents, each topic can store additional data in name/value pairs.

Topic data is normally not visible when you view a topic (except for a small table at the bottom of the topic - dependent on the used skin). Topic data works "behind the scenes" and facilitates searches, reports and custom displays.

Topic data, or better: structured data, can be used in many ways. The Foswiki Support questions serves as a demonstration how topic data can be used:

To create a complete list of "Support Questions" topics
To show a subset of all questions that have not been answered yet
To display the title and subject of each question

Another uses for structured data could be:

Create a list of all contributions of one particular author
Create a quick report of all employee names and phone numbers
Create a software documentation repository
Create and track tasks
Create a bug tracker

To work with topic data, you will need 3 things:

The data definition, specified in a table in a "data form" topic. The table lists all fields and their types - see The data definition.
The web's WebPreferences needs to list the form in the WEBFORMS setting - see Enabling forms.
The form must get connected to the topic - see Adding a form to a topic.

Then you are ready to enter data values for each topic.

Optional further steps are:

Create a template topic so that new topics based on that template have the form already added - see Template topics.
Build an HTML form to create new topics based on that template topic, or to pass field values from the HTML form to the data form.
Expand form fields into the text using the %FORMFIELD% macro - see VarFORMFIELD
Create a custom View Template to display your data. See SkinTemplates and AutoViewTemplatePlugin.
Build a search to create a report from the stored data

For a step by step tutorial, see the FAQ How can I create a simple data form based application?.

Sometimes new users with a web programming background are confused how "data forms" and "HTML forms" are related. They are not related. But you'll see later on that you can use web forms to pass data to a topic data form.

The data definition

Topics can store data as name/value pairs, or form fields. The attributes of each form field are specified in the data definition, which is an ordinary topic.

The data form topic

The data definition is defined in a TML table and looks like this:

| *Name*  | *Type* | *Size* | *Values* | *Description* | *Attributes* | *Default* |
| TopicTitle | text | 100 | | Title of the topic | H | |
| Version | select | 1 | Version in SVN,Foswiki 1.1.4,Foswiki 1.1.3,Foswiki 1.1.2,Foswiki 1.1.1 | | | |
| Status | select | 1 | Asked,More info required,Answered | Mandatory status | M | Asked |
| Related Topics | textboxlist | | | | | |

In the next section we'll go into the details of the table contents.

The name of the data form topic usually ends with "Form". For example, the form topic for the Support Questions is named "QuestionForm". The form topic can be placed in any web, but usually this is in the same web as the topics that will be using it.

Topic types
You could consider the data form topic as the data type. For instance, all topics that use the form QuestionForm are topics of type "Question".

A useful core feature of Foswiki 1.1 is the automatic selection of view and edit templates based on the name of the form attached to a topic. With this we are really starting build up a topic as something resembling a typed object: its form name being the type identifier, and its form+templates as the details of its implementation. See AutoViewTemplatePlugin for details of this feature.

General Notes:

The topic definition is not read when a topic is viewed.
Form definition topics can be protected in the usual manner, using AccessControl, to limit who can change the form definition and/or individual value lists. Note that view access is required to be able to edit topics that use the form definition, though view access to the form definition is not required to view a topic where the form has been used.

The form table

A form is to a web as a table is to a database. -- Andrew Steele

The data form table is a kind of spreadsheet:

Each row of the table specifies one form field
The table header defines what attributes of the form fields need to be specified

The data form table contains up to 7 columns. The first three (Name, Type and Size) are required, the others are optional.

Name
Type
Size
Values
Description
Attributes
Default

Header row

Data form tables often have an optional header row as a useful reminder of the column names:

| *Name*  | *Type* | *Size* | *Values* | *Description* | *Attributes* | *Default* |

Form field rows

One form field is defined by each row in the table.

Name	The name of the form field; must be unique for that form.
Type	The data type: text, date, single or multi-value, labels (read-only). The type also defines how form field data can be entered in the edit screen, such as text field or radio buttons.
Size	The input size of the form field inputs on the edit screen.
Values	For checkboxes, radio buttons and dropdown lists: predefined input to select from. More advanced: this can be a dynamically generated list of values.
Description	A message that describes the field.
Attributes	Attributes controlling how the field is displayed.
Default	if there is a `default` column, its value (or lack of value) is used as the default for new fields. This allows you to default the value of `select`, `radio` and `checkbox` to something other than the first `values` element

Form field attributes

Name

The name of the form field.

Names have to be unique for each data definition.
A very few field names are reserved. If you try to use one of these names, Foswiki will automatically append an underscore to the name when the form is used. But do not use the field name undefined (or any variant of that name, such as UnDefined), as that name is reserved for use in search queries.
You can space out the title of the field, and it will still find the topic e.g. Aeroplane Manufacturers is equivalent to AeroplaneManufacturers.
If a label field has no name, it will not be shown when the form is viewed, only when it is edited.
Field names can in theory include any text, but you should stick to alphanumeric characters.
- Leading and trailing spaces do not matter.
- The field name used for a select, checkbox or radio field, can also be a topic-name used to fetch values from that topic.
- You can use [[...]] double bracket links to link to a topic/field name that is not a wiki word.
- You can also use square bracket links in the form of [[topicname][field name]] to use a field name other than the topic name. ie. [[ComputerManufacturers][Made by]]
(In this example, the topic ComputerManufacturers is used to fetch values for the Madeby field.)
- Spaces will be stripped from the field name. ie: Made by becomes Madeby.
- Releases prior to Foswiki 2.0 also stripped characters other than A-Z, a-z, 0-9 and _. So a field named Fühler would be stored as Fhler. The same field name on Foswiki 2.0 is stored as Fühler. (See System.UpgradeGuide#DataForms_Applications for more information.}

Type

The data type defines the kind of input: text, date, single or multi-value, or labels (read-only). This is done by setting the type of interface control on the edit screen: checkbox, radio button, text field, and so on.

The control appearance is also specified by size and (initial) value. More on those attributes below.

Type	Description	Size attribute	Value attribute	Modifiers
`checkbox`	One or more checkboxes.	How many checkboxes will be displayed on each line.	A comma-separated list of item labels.	`checkbox+buttons` will add Set and Clear buttons to the basic `checkbox` type. `checkbox+values` allows the definition of values that are different to the displayed text.
`date`	A single-line text box and a calendar icon button next to it; clicking on the button will bring up a calendar from which the user can select a date. The date can also be typed into the text box.	The text box width in characters.	The initial text (unless `default` column exists).
`label`	Read-only label text.		The text of the label.
`radio`	Like `checkbox` except that radio buttons are mutually exclusive; only one can be selected.			`radio+values` allows the definition of values that are different to the displayed text.
`select`	A select box / dropdown.	A fixed size for the box (e.g. `1`, or a range e.g. `3..10`. To get a dropdown, use size `1`. If you specify a range, the box will never be smaller than 3 items, never larger than 10, and will be 5 high if there are only 5 options.	A comma-separated list of options for the box.	`select+multi` turns multiselect on for the select, to allow Shift+Click and Ctrl+Click to select (or deselect) multiple items. `select+values` allows the definition of values that are different to the displayed text. You can combine these modifiers e.g. `select+multi+values`
`text`	A one-line text field.	The text box width in number of characters.	The initial (default) content when a new topic is created with this form definition (unless `default` column exists).
`textarea`	A multi-line text box.	Size in columns x rows, e.g. `80x6`; default size is 40x5.	The initial text (unless `default` column exists).

Example of select+values

A formfield definition like:

| Field 9 | select+values | 1 | One, Two=2, Three=III, Four | Various values formats |

displays as:

The generated HTML code reveals that the form values differ from the option labels:

<select name="Field9" size="1" class="foswikiSelect">
<option value="One" class="foswikiOption">One</option>
<option value="2" class="foswikiOption">Two</option>
<option value="III" class="foswikiOption">Three</option>
<option value="Four" class="foswikiOption">Four</option>
</select>

Extending the range of form data types

Some plugins add data types. For instance, JQueryPlugin's sub-plugin JQueryFarbtastic adds the data type color:

| *Name*  | *Type* | *Size* |
| MyColor | color | 10 |

Example (click in the field to view):

Note to extension developers
Such extended data types are single-valued (can only have one value) with the following exceptions:

any type name starting with checkbox
any type name with +multi anywhere in the name

Types with names like this can both take multiple values.

Size

The input size of the form field inputs on the edit screen. The size acts a bit different for each type - see the Type table above.

Values

For checkboxes, radio buttons and dropdown lists: predefined input to select from. More advanced: this can be a dynamically generated list of values.

The field value will be used to initialize a field when a form is created (unless default column exists), unless specific values are given by the topic template or query parameters. The first item in the list for a select or radio type is the default item. For label, text, and textarea fields the value may also contain commas. checkbox fields cannot be initialized through the form definition.
Leading and trailing spaces do not matter.
Field values can also be generated through a FormattedSearch, which must yield a suitable table as the result.
Macros in the initial values of a form definition get expanded when the form definition is loaded.
- If you want to use a | character in the initial values field, you have to precede it with a backslash, thus: \|.
  - \| escaping is _only active in the Values column. It is not usable elsewhere. Use &vbar; or | in other columns.
- You can use <nop> to prevent macros from being expanded.
- The Format tokens can be used to prevent expansion of other characters.

Default

If this column exists in the form definition, then it will be used to determine the default value for a field. It over-rides all defaulting from the Values column.

How to retrieve values from other topics

If you have rows defined like this:

| *Name*  | *Type* | *Size* |
| AeroplaneManufacturers | select | |

... Foswiki will look for the topic AeroplaneManufacturers to get the possible values for the select field.

The Values column must be empty.

The AeroplaneManufacturers topic must contain a table, where each row of the table describes a possible value. The table only requires one column, Name. Other columns may be present, but are ignored.

For example:

| *Name* |
| Routan |
| Focke-Wulf |
| De Havilland |

How to set field values using a macro

A powerful way to populate selectable field values is by using SEARCH to generate a comma-separated list of values.

For example, to create a list of documentation topics whose name contain "Wiki", you write:

| *Name*  | *Type* | *Size* | *Values* |
| Subject | select+multi | 10 | ,%SEARCH{ \
   "Wiki" \
   scope="topic" \
   web="%SYSTEMWEB%" \
   nonoise="on" \
   type="regex" \
   format="$topic" \
   separator=", " \
}% |

Note the use of the backslash to be able to write the macro in a more readable way using multiple lines. The comma just before the SEARCH means "empty value" to make it possible to select none.

Sometimes you have a topic with a bullet list that can be used as selectable values. The rather advanced SEARCH expression would be:

| Subject | select+multi | 10 | ,%SEARCH{ \
 "   *\s*.*?" \
 topic="QuestionSubjectCategory" \
 type="regex" \
 multiple="on" \
 casesensitive="on" \
 nonoise="on" \
 separator="," \
 format="$pattern(.*   \*\s*([^
]*).*)" \
}% | | |

You are not expected to write these kind of search expressions yourself, but if you like you can find more of these in Search Pattern Cookbook.

Fields and linefeeds
Some browsers may strip linefeeds from text fields when a topic is saved. If you need linefeeds in a field, make sure it is a textarea.

Description

Description of the field. Sometimes used in help information.

Attributes

Whether the field is mandatory or hidden when viewed.

`H`	Indicates that this field should not be shown in view mode. However, the field is available for editing and storing information.
`M`	Indicates that this field is mandatory. The topic cannot be saved unless a value is provided for this field. If the field is found empty during topic save, the user is presented with an error message. Mandatory fields are indicated by an asterisk next to the field name.

Multiple attributes can be entered, separated by spaces:

| TopicTitle | text | 100 | | | H M |

Enabling forms

Before connecting topics to a data definition, the definition must be enabled in the Web's WebPreferences topic.

This is done by adding the form topic name to the WEBFORMS setting. The setting accepts a comma-separated list of form topics:

   * Set WEBFORMS = BugForm, FeatureForm, Books.BookLoanForm, %USERSWEB%.UserForm

As you can see, form topics located in other webs can be added by using their web prefix.

You have to list the available form topics explicitly. You cannot use a SEARCH to define WEBFORMS.

Adding a form to a topic

With WEBFORMS enabled, a form can be added or changed on the edit screen.

Manual operation

Add a form: click the "Add form" button and select one of the forms in the list.
Switching or removing a form: click the "Change form" button and select another one, or none, from the list.

Automatically adding a form to a new topic

Using a template topic

If you don't want users to select a form themselves, this step can be automated by using a template topic that has the form connected.

For example, Question topics on foswiki.org are created using QuestionTemplate:

%META:FORM{name="QuestionForm"}%
%META:FIELD{name="TopicTitle" attributes="H" title="TopicTitle" value=""}%
%META:FIELD{name="Subject" attributes="" title="Subject" value=""}%
%META:FIELD{name="Status" attributes="M" title="Status" value="Asked"}%

In the template initial field values can be set, like the "Status" field in this example.

Using the edit template

If you will use only one form in a web, you can also modify the web's WebTopicEditTemplate topic to contain the form reference.

Using url parameters

You can also pass the formtemplate parameter to the edit (not save) URL. Initial values can then be provided in the URLs or as form values.

For example:

%SCRIPTURL{edit}%/%WEB%/KnowledgeTopicAUTOINC00001?formtemplate=MyForm;Subject=FAQWhatIsWikiWiki;Category=One;Category=Two;action=form

XxxYyyAUTOINC00001: creates a new auto-numbered topic name with base "XxxYyy" - see automatically generate unique topic names
formtemplate: specifies the form topic, like: formtemplate=MyForm
form values: name=value, like: Subject=FAQWhatIsWikiWiki
form values with multiple values like checkboxes: name=value1;name=value2, like: Category=One;Category=Two
action=form: shows the topic data form and hides the topic text form

Using a web form to create a topic and pass data

If you want to create+save a topic instead of bringing up the edit screen, you must use a web form.

The same parameters as above can be set in HTML (hidden) form fields:

<form name="newtopic" action="%SCRIPTURLPATH{"save"}%/Sandbox/" method="post">
<input type="hidden" name="formtemplate" value="SimpleForm" />
<input type="hidden" name="topic" value="KnowledgeTopicAUTOINC00001" />
<input type="hidden" name="Subject" value="FAQWhatIsWikiWiki" />
<input type="hidden" name="Category" value="One" />
<input type="hidden" name="Category" value="Two" />
<input type="submit" class="foswikiSubmit" value="Create topic" />
</form>

Creates: For an overview of web form parameters, see CGI and Command Line Scripts.

Changing a form

You can change a form definition, and Foswiki will try to make sure you don't lose any data from the topics that use that form.

If you change the form definition, the changes will not take affect in a topic that uses that form until you edit and save it.
If you add a new field to the form, then it will appear next time you edit a topic that uses the form.
If you delete a field from the form, or change a field name, then the data will not be visible when you edit the topic (the changed form definition will be used). If you save the topic, the old data will be lost (though thanks to revision control, you can always see it in older versions of the topic)
If two people edit the same topic containing a form at exactly the same time, and both change fields in the form, Foswiki will try to merge the changes so that no data is lost.

Searching in form data

The best way to search in form data is using the structured query language in the SEARCH macro.

As an example, the search used on Foswiki Support questions is:

%SEARCH{
 "QuestionForm.Status='Asked'"
 type="query"
 excludetopic="QuestionTemplate"
 web="%WEB%"
 format="   * [[$web.$topic][$formfield(TopicTitle)]] $formfield(Subject)"
 order="formfield(Extension)"
 nonoise="on"
}%

For a step by step tutorial, see the FAQ How can I create a simple data form based application?.

See SEARCH for an overview of SEARCH parameters, and Query Search for the query language specifics.