Unlike other supported “subsettable” formats, this ingest mechanism requires 2 files: the CSV raw data file proper and an SPSS Setup file ("control card") with the data set metadata. In the future, support for other data definition formats may be added (STATA, SAS, etc.). As always, user feedback is welcome.

The supported SPSS command syntax:

Please note that it is not our goal to attempt to support any set of arbitrary SPSS commands and/or syntax variations. The goal is to enable users who do not own proprietary statistical software to prepare their raw data for DVN ingest, using a select subset of SPSS data definitional syntax.

(In addition to its simplicity and popularity, we chose to use the SPSS command syntax because DVN already has support for the SPSS .SAV and .POR formats, so we have a good working knowledge of the SPSS formatting conventions).

The following SPSS commands are supported:

DATA LIST
VARIABLE LABELS
NUMBER OF CASES
VALUE LABELS
FORMATS (actually, not supported as of now -- see below)
MISSING VALUES

We support mixed cases and all the abbreviations of the above commands that are valid under SPSS. For example, both "var labels" and "Var Lab" are acceptable commands.

Individual command syntax.

1. DATA LIST

An explicit delimiter definition is required. For example:

DATA LIST LIST(',')

specifies ',' as the delimiter. This line is followed by the '/' separator and variable definitions. Explicit type definitions are required. Each variable is defined by a name/value pair VARNAME
(VARTYPE) where VARTYPE is a standard SPSS fortran-type definition.

Note that this is the only required section. The minimum amount of metadata required to ingest a raw data file is the delimiter character, the names of the variables and their data type. All of these are defined in the DATA LIST section. Here’s an example of a complete, valid control card:

DATA LIST LIST(’,’) /
CASEID (f) NAME (A) RATIO (f)
.

It defines a comma-separated file with 3 variables named CASEID, NAME and RATIO, two of them of the types numeric and one character string.

Examples of valid type definitions:

A8 8 byte character string;
A character string;
f10.2 numeric value, 10 decimal digits, with 2 fractional digits;
f8 defaults to F8.0
F defaults to F.0, i.e., numeric integer value
2 defaults to F.2, i.e., numeric float value with 2 fractional digits.

The following SPSS date/time types are supported:

type format

DATE yyyy-MM-dd
DATETIME yyyy-MM-dd HH:mm:ss

The variable definition pairs may be separated by any combination of white space characters and newlines. Wrapped-around lines must start with white spaces (i.e., newlines must be followed by spaces). The list must be terminated by a line containing a single dot.

Please note, that the actual date values should be stored in the CSV file as strings, in the format above. As opposed to how SPSS stores the types of the same name (as integer numbers of seconds).

2. VARIABLE LABELS

Simple name/value pairs, separated by any combination of white space characters and newlines (as described in section 1 above). The list is terminated by a single dot.

For example:

VARIABLE LABELS
CELLS "Subgroups for sample-see documentation"
STRATA "Cell aggregates for sample”
.

3. NUMBER OF CASES (optional)

The number of cases may be explicitely specified. For example:

num of cases 1000

When the number of cases is specified, it will be checked against the number of observations actually found in the CSV file, and a mismatch would result in an ingest error.

4. VALUE LABELS

Each value label section is a variable name followed by a list of value/label pairs, terminated by a single "/" character. The list of value label sections is terminated by a single dot.

For example,

VALUE labels
FOO 0 "NADA"
1 "NOT MUCH"
99999999 "A LOT"
/
BAR 97 "REFUSAL"
98 "DONT KNOW"
99 "MISSING"
/
.

5. FORMATS

This command is actually redundant if you explicitly supply the variable formats in the DATA LIST section above.

<NOTE:> It appears that the only reason the "FORMATS" command exists is that DATA LIST syntax does not support explicit fortran-style format definitions when fixed-field data is defined. So it is in fact redundant when we're dealing with delimited files only</NOTE:>

Please supply valid, fortran-style variable formats in the DATA LIST section, as described above.

6. MISSING VALUES

This is a space/newline-separate list of variable names followed by a comma-separated list of missing values definition, in parentheses. For example:
INTVU4 (97, 98, 99)
The list is terminated with a single dot.

An example of a valid MISSING VALUES control card section:

MISSING VALUES
INTVU4 (97, 98, 99)
INTVU4A ('97', '98', '99')
.

An example of a control card ready for ingest:

data list list(',') /
CELLS (2) STRATA (2) WT2517 (2)
SCRNRID (f) CASEID (f) INTVU1 (f)
INTVU2 (f) INTVU3 (f) INTVU4 (f)
INTVU4A (A)
.
VARIABLE LABELS
CELLS "Subgroups for sample-see documentation"
STRATA "Cell aggregates for sample-see documenta"
WT2517 "weight for rep. sample-see documentation"
SCRNRID "SCREENER-ID"
CASEID "RESPONDENT'S CASE ID NUMBER"
INTVU1 "MONTH RESPONDENT BEGAN INTERVIEW"
INTVU2 "DAY RESPONDENT BEGAN INTERVIEW"
INTVU3 "HOUR RESPONDENT BEGAN INTERVIEW"
INTVU4 "MINUTE RESPONDENT BEGAN INTERVIEW"
INTVU4A "RESPONDENT INTERVIEW BEGAN AM OR PM"
.
VALUE labels
CASEID          99999997 "REFUSAL"
                  99999998 "DONT KNOW"
                  99999999 "MISSING"
                  /
INTVU1          97 "REFUSAL"
                  98 "DONT KNOW"
                  99 "MISSING"
                  /
INTVU2          97 "REFUSAL"
                  98 "DONT KNOW"
                  99 "MISSING"
                  /
INTVU3          97 "REFUSAL"
                  98 "DONT KNOW"
                  99 "MISSING"
                  /
INTVU4          97 "REFUSAL"
                  98 "DONT KNOW"
                  99 "MISSING"
                  /
INTVU4A         "97" "REFUSAL"
                  "98" "DONT KNOW"
                  "99" "MISSING"
                  "AM" "MORNING"
                  "PM" "EVENING"
.
MISSING VALUES
CASEID (99999997, 99999998, 99999999)
INTVU1 (97, 98, 99)
INTVU2 (97, 98, 99)
INTVU3 (97, 98, 99)
INTVU4 (97, 98, 99)
INTVU4A ('97', '98', '99')
.

NUMBER of CASES 2517

DATA FILE.

Data must be stored in a text file, one observation per line. Both DOS and Unix new line characters are supported as line separators. On each line, individual values must be separated by the delimiter character defined in the DATA LISTsection. There may only be exactly (NUMBER OF VARIABLES - 1) delimiter characters per line; i.e. character values must not contain the delimiter character.

QUESTIONS, TODOS:

Is there any reason we may want to support RECODE command also?

--- comments, suggestions are welcome! ---

Guides

Help for Users, Installers, and Developers

Appendix