. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
by Wendy Duff, School of Information Studies, University of Toronto
This article reports on a research study that tested the effect of statements of "literary warrant" on lawyers, auditors and information specialists' evaluations of a set of functional requirements for electronic evidence. It found that legal statements can increase the rating of importance of some of the functional requirements. Its results also provided evidence that differences in subjects professional backgrounds and their computer knowledge can affect the ratings of importance the subjects gave to the functional requirements.
Organizations maintain records to meet the legal, fiscal, and administrative obligations that are dictated by society. Traditionally records managers have emphasized the need to tie records management programs to legal retention requirements. Over the last five years, a number of research projects concerned with the management of electronic records have reaffirmed the importance of grounding one's records programs on a firm legal foundation. For example:
The Pittsburgh Project also postulated that the laws, regulations, case law, information technology standards, auditing standards and best practices promulgated by lawyers, auditors, information specialists, business managers, record managers, and the medical profession could be used to build a strong case for promoting specifications for keeping reliable electronic records. The project staff believed it was important to build this case out of a recognition that the specifications for preserving electronic evidence may be undervalued by organizations. They also suggested that other professionals would be more inclined to accept the functional requirements that the Project had identified, if the requirements were tied to laws, regulations and best practices recommended by other professional associations. They stated that "if professionals in our society were made more aware of the functional requirements for recordkeeping as expressed in recommended practices of their own profession (which are themselves grounded in law), they would be more inclined to take responsibility for the adequacy of their recordkeeping practices."
The project team suggested that archivists and/or records managers could use the statements from laws, regulations, standards, etc., as "literary warrant," that is, as proof or justifi-cation that organizations and individuals must adhere to the requirements because they are based on practices established by their own professions or industry. The Project suggested that decision-makers would value the requirements more, if the requirements were tied to literary warrant than if they were presented on their own. This article reports on a study that tested that suggestion. Specifically it asked the following questions:
The study comprised three different stages. In the first stage of the study, the investigator compiled a list of authoritative sources which relate to professional practices in the law, auditing and information technology fields and dictate requirements for recordkeeping. Nine reviewers in the fields of auditing, law, and information technology (three individuals in each field) rated the authority of the sources from which the warrant was drawn as a check on the credibility of the sources. The authority of each source was scored on a scale of 0-3, with 3 having a great deal of authority and 0 having none.
In the second stage, each authoritative source was scanned for relevant passages that illustrate the functional requirements. Each passage was classified according to the professional group to which the source related and the functional requirement that it supported. Three members of the Pittsburgh project team (Richard Cox, Ken Sochats, and David Bearman) evaluated each passage for its relevance to, and its support for, a functional requirement for evidence.
In the final stage, four research instruments were created. The investigator created the research instruments by first compiling four lists: 1.) just the University of Pittsburgh Project's functional requirements; 2.) the functional requirement augmented with statements of legal warrant; 3.) the functional requirements augmented with statements of auditing warrant; and finally, 4.) the functional requirements augmented with statements of information technology warrant. The investigator then randomly selected a functional requirement from the first list and assigned it to the first research instrument. A functional requirement and its accompanying auditing warrant was randomly chosen from the second list and assigned to the first research instrument. If the functional requirement already existed in the research instrument (having been taken from another list), a new requirement was selected. A functional requirement and its accompanying legal warrant were randomly chosen from the third list and assigned to the first research instrument. If the functional requirement already existed in the research instrument (having been taken from another list), a new requirement was selected. Then a functional requirement and its accompanying information technology warrant was randomly chosen from the fourth list and assigned to the research instrument. If the functional requirement already existed in the research instrument (having been taken from another list), a new requirement was selected. This process continued until the first research instrument had a complete set of functional requirements (1-20) with five functional requirements being presented on their own, five being accompanied by auditing warrant, five being accompanied by legal warrant, and five being accompanied by information technology warrant.
The second research instrument was created following the same procedure, except that warrants selected for the first research instrument were not included in the selection for second instrument. The creation of third and fourth instruments also followed the same procedure.
Each of the research instruments was presented to one of four different groups of subjects. Each group of subjects consisted of five lawyers, five auditors and five information specialists. Semi-structured interviews were conducted at the subjects' workplace:
The primary purpose of this study was to test the influence of literary warrant on the scores given to the functional requirements by lawyers, auditors and information specialists. It hypothesized that a person's judgment of a functional requirement would be significantly higher when a functional requirement was accom-panied by warrant than when it was presented on its own. As previously noted, the Pittsburgh Project suggested that it was important to build a case for the requirements because on their own the requirements would be undervalued. Before examining the evidence pertaining to warrant, the investigator first sought to determine how important the subjects thought the functional requirements were.
Functional Mean St Dev Minimum Maximum Requirements Accurate 8.55 1.00 5.00 9.00 Available 8.45 1.21 3.00 9.00 Consistent 8.05 1.36 3.00 9.00 Inviolate 8.02 1.44 3.00 9.00 Compliant 7.97 1.35 3.00 9.00 Authorized 7.95 1.38 400 9.00 Documented 7.87 1.20 4.00 9.00 Identifiable 7.60 1.40 4.00 9.00 Redactable 7.48 1.75 3.00 9.00 Coherent 7.42 1.59 3.00 9.00 Meaningful 7.35 1.35 4.00 9.00 Renderable 7.28 1.42 4.00 9.00 Implemented 7.23 1.69 2.00 9.00 Auditable 7.17 1.78 3.00 9.00 Comprehensive 7.15 1.68 3.00 9.00 Exportable 7.13 2.01 2.00 9.00 Evidential 6.98 1.68 4.00 9.00 Assigned 6.93 1.81 1.00 9.00 Understandable 6.72 1.68 1.00 9.00 Removable 6.45 1.97 1.00 9.00
The Functional Requirements
The evaluations of the functional requirements with and without warrant were compiled and their means were analyzed. Table 1 presents the mean scores, their variations and the mini-mum and maximum scores given to each requirement.
The evaluations of the individual functional requirements varied, with the average score given to Accurate being a high of 8.55 with a standard deviation of 1.00, and the average score given to Removable being a low of 6.45 with a standard deviation of 1.97.
The frequency and cumulative frequency of each score given to each functional requirement was also tabulated. On the whole, the ratings were high, with only 76 of the 1200 scores being a rating of less than 5. Only four requirements (Understandable, Removable, Exportable, and Evidential) had more than 10% of their evaluations rated as less than 5, on a 9 point scale, while twelve of the twenty requirements received scores of 8 or 9 for more than 50% of their evaluations. Accurate, evaluated as the most important requirement, obtained a score of 9, for 78.3% of its evaluations.
To obtain a comprehensive overview of the average scores given to all the functional requirements without warrant and with legal, auditing and information technology warrant, the means of each functional requirement under each condition were computed. Table 2 contains the average scores given to each requirement divided into the type of warrant that accompanied it. The condition that received the highest average rating for each functional requirement is marked with a plus sign (+).
Functional Without With With With IT Requirement Warrant Legal Auditing Warrant Warrant Warrant Compliant 8.13+ 8.07 7.93 7.73 Documented 8.20+ 7.4 7.93 7.93 Assigned 6.87 8.07+ 5.93 6.87 Implemented 7.0 7.53 7.8+ 6.6 Consistent 7.27 8.47+ 8.27 8.2 Comprehensive 7.0 7.87+ 6.4 7.33 Identifiable 7.53 7.93+ 7.87 7.07 Accurate 8.33 8.73+ 8.6 8.53 Understandable 6.87+ 6.80 6.6 6.6 Meaningful 6.67 7.47 7.47 7.8+ Authorized 7.9 8.87+ 7.6 7.5 Inviolate 8.2+ 7.47 8.2+ 8.2+ Coherent 7.93+ 7.33 7.6 6.8 Auditable 6.40 8.00+ 7.40 6.87 Removable 5.93 6.73 6.80+ 6.33 Exportable 6.87 6.93 8.13+ 6.60 Available 8.53 8.73+ 8.13 8.4 Renderable 7.07 7.20 7.33 7.53+ Evidential 6.20 7.47+ 7.00 7.27 Redactable 7.27 7.20 8.20+ 7.27
Of the twenty functional requirements, the highest average score for nine of the requirements was attained when they were accompanied by legal warrant. Four functional requirements received their highest average scores when they were accompanied by auditing warrant and another four requirements received the highest average scores when they were accompanied by no warrant. Only two functional requirements received their highest average scores when they were accompanied by informa-tion technology warrant. There was one tie among the highest average scores that one functional requirement received. Inviolate received the same score when it was accompanied by auditing warrant, when it was accompanied by information technology warrant, and when it was presented without warrant.
Seven functional requirements received their lowest average scores when they were accompanied by no warrant. The lowest scores for six functional requirements were attained when they were accompanied by information technology warrant. Three requirements received their lowest average score when they were accompanied by auditing warrant and another three received their lowest average scores when they were accompanied by legal warrant. There was one tie between the lowest average score that a functional requirement attained when it was accompanied by information technology and auditing warrant.
To discover if the scores given to the functional requirements accompanied by different types of warrant were significantly different, an Analysis of Variance including the Scheffe test was conducted. Only two requirements, Assigned and Authorized had any significant differences in their evaluations, and both received their highest mean rating when they were accompanied by legal warrant.
An Analysis of Variance with repeated measures was conducted to obtain a better understanding of the overall effect of warrant. The four groups of subjects were analyzed separately and the data provided by each group of subjects indicated a strong relationship or synergy between the warrant and the func-tional requirement, which was significant at the .01 level. In some cases the presence of warrant can make a significant difference, but in other cases it is the strong relationship between a particular piece of warrant and a specific functional requirement that is significant.
To discover if the different professional groups were affected by the warrant, the data were divided by professional group and an Analysis of Variance was conducted. This method of analysis resulted in extremely small samples (only five subjects in each category). With these small sample sizes, only lawyers showed any significant differences in the ratings they assigned the requirements accompanied by different types of warrant. The presence of warrant significantly affected the lawyers' evaluations of four of the twenty requirements, Assigned, Implemented, Authorized and Auditable. The presence of warrant did not cause any statistically significant differences in the evaluations provided by the auditors or information specialists.
Persuasion research suggests that when the cognitive effort is too great to interpret a message, people will use simple decision rules that depend upon persuasion cues, e.g., source credibility, with message validity. Research also suggests that people with a strong working-knowledge are less likely to be persuaded by heuristics or simple persuasion cues, such as the credibility of a message, than people with less working-knowledge. The lawyers in the study had far less knowledge of computers and electronic records, and therefore they probably would have needed greater cognitive effort to interpret the functional requirements than the information specialists or the auditors. The finding that warrant was more effective in influencing the opinions of lawyers than information specialists or auditors is not surprising considering the conclusions of previous persuasion research.
An Analysis of Variance was performed to discover if there were significant differences in the scores given by the different professional groups. The differences in the rating given to the requirements by the different professional groups were statistically significant, at the .05 level, for only one functional requirement, Comprehensive. The highest average ratings were provided by auditors. Authorized showed a strong, but not quite significant difference, in the scores given by the different professional groups, with information specialists giving the highest average score.
To obtain a overview of the average scores provided by the different professional groups, the means of each functional requirement given by each professional group were computed. Table 3 contains the average scores given to each requirement divided into the professional group that provided it.
Functional Information Lawyers Auditors Requirement Specialists Compliant 8.00 7.70 8.20 Documented 8.25+ 7.65 7.70 Assigned 6.95 6.60 7.25 Implemented 7.60+ 7.05 7.05 Consistent 8.30 7.50 8.35 Comprehensive 7.05 6.30 8.10 Identifiable 8.00+ 7.30 7.50 Accurate 8.55 8.60+ 8.50 Understandable 7.05+ 7.00 6.10 Meaningful 7.65+ 7.25 7.15 Authorized 8.25+ 7.35 8.20 Inviolate 8.15 7.65 8.25 Coherent 7.40 7.40 7.45 Auditable 7.25 6.60 7.65 Removable 6.45 6.50+ 6.40 Exportable 7.45+ 7.35 6.60 Available 8.15 8.60+ 8.60 Renderable 7.50+ 7.15 7.20 Evidential 7.30+ 6.50 7.15 Redactable 7.75+ 7.60 7.10
The professional group that received the highest average rating for each functional requirement is marked with a plus sign (+).
Of the twenty functional requirements, ten received their highest mean scores from information specialists, while seven received their highest mean scores from auditors. Lawyers provided the highest mean scores for only four functional require-ments. There was one tie between the highest average scores that a functional requirement received when it was evaluated by lawyers and when it was evaluated by information specialists.
On the other hand, lawyers provided the lowest mean scores for eleven of the twenty evaluations. Only one functional requirement received its lowest mean score from information specialists. There was one tie between the lowest average scores that one functional requirement received when it was evaluated by lawyers and when it was evaluated by auditors. There was also one tie between the lowest average scores that another functional requirement received when it was evaluated by lawyers and when it was evaluated by information specialists.
On the whole, information specialists and auditors seemed to have more appreciation for the functional requirements than the lawyers in the study. Lawyers provided the highest average score for only two requirements and gave the lowest average score to eleven functional requirements. Perhaps individuals with a high degree of computer knowledge, like the information specialists and auditors in this study, can appreciate more fully the importance of the requirements.
The ranking of the functional requirements also suggest that the different professional groups value the requirements differently. Auditors rated Comprehensive as the seventh most important requirement, while information specialist ranked it as the eighteenth most important requirement, and lawyers ranked it last. Information specialists ranked Authorized as the second most important requirement, auditors rated it as fifth, and lawyers ranked it as the ninth most important requirement. Redactable, however was ranked as the sixth most important requirement by lawyers, the ninth most important requirement by information specialists, and the fifteenth most important requirement by auditors.
Auditors may rate Comprehensive as more important than the other professional groups because their training and experience is related often to auditing financial records. The concept that a system should capture all financial transactions is well established. The auditors in this study may have equated the requirement, Comprehensive, to the capturing of financial trans-actions, rather than all transactions. The professional background of a subject also strongly affected the scores given to Authorized, with information specialists ranking this requirement higher than lawyers or auditors. Many of the techniques for authorizing data, such as digital signatures or passwords, have been developed to fulfill requirements for data and systems security. Perhaps information specialists understand the close connection between this requirement and security measures, and therefore ranked it as more important than the other professional groups. Lawyers ranked Redactable higher than the other professional groups suggesting that lawyers have a greater appreciation for the need to mask confidential information.
The differences in the ratings given by the three professional groups may be due to differences in professional perspective, or they may have arisen from differences in the subjects' background. To discover if there was a relationship between these variables and the evaluations given to the functional require-ments, data on the subjects' background were collected and correlation analysis conducted. Correlations tended to be small and not significant, with the exception of the subjects' knowledge of, and experience with, computers. The ratings given to the requirements Comprehensive, Authorized and Auditable corre-lated positively with the subjects' computer knowledge. The ratings given to Documented correlated positively with the years the subjects had used computers.
Subjects may gain a greater appreciation for the importance of documenting systems, capturing all records, maintaining audit trails of a record's use, and ensuring only authorized individuals create records, as they gain knowledge of, and experience with, computers. This does not suggest that people will suddenly understand the need for reliable evidence as they become com-puter literate. Rather, it suggests that some of the requirements needed to develop trustworthy information systems, such as having systems documentation and ensuring adequate security measures, are similar to some of the requirements for having reliable evidence.
Warrant can successfully increase the acceptance of the functional requirements in some cases. The scores given to certain functional requirements, in particular Assigned and Authorized, were affected significantly by the presence of legal warrant. Legal warrant appears to have the greatest influence, and lawyers appear to be more influenced by warrant than information specialists or auditors. What is not known is whether legal warrant had its greatest influence because legal sources have the greatest authority, or whether legal warrant had the greatest influence because lawyers, on the whole, were the most influenced by it.
Analyzing the functional requirements as a whole, rather than individually, provides evidence that there is a significant relationship between the functional requirements and the warrant. Developing a warrant for the functional requirements is important, although it may be more important for increasing the acceptance of some requirements than others, and it may be more effective in influencing the opinion of some professional groups than others. For example, it may have a greater effect on influencing the opinions of people with less computer knowledge. Although warrant may not significantly increase the acceptance of individual functional requirements, its presence may have enhanced the credibility of all the requirements. This study supports, in part, the belief that warrant may be an important tool that archivists and records managers can use to influence the design of recordkeeping systems and ensure that electronic records are captured and maintained in the future. More research is needed to verify this conclusion.
Warrant is not the only solution needed to ensure the require-ments for reliable evidence become accepted and incorporated into the design of systems. The subject's computer knowledge and experience showed a mild positive correlation with the rating of importance of some functional requirements and the professional background strongly affected the rating of two requirements. The study also showed that a strong working-knowledge of computers may decrease the influence of warrant. There is a strong relationship, or synergy, between the functional requirements and the warrant which needs further study. Further research on warrant, including the effect of different types of warrant on other professional groups and the effect of the relationship between the warrant and the requirements, needs to be conducted. When this research is undertaken the results may point to new strategies needed to ensure that archivists and records managers become involved in the design of record-keeping systems that capture and preserve electronic records.
As noted in the introduction, the University of Pittsburgh responded to the NHPRC's electronic records research agenda by conducting a research project to develop a set of functional requirements for recordkeeping. These requirements are system independent and could be implemented in either a manual, electronic or hybrid system.
The nineteen requirements for recordkeeping are grouped into three different categories:
1. Compliant
2. Responsible
3. Implemented
4 Consistent
5. Comprehensive
6. Identifiable
7. Complete
7a. Accurate
7b. Understandable
7c. Meaningful
8. Authorized
9. Preserved
9a. Inviolate
9b. Coherent
9c. Auditable
10. Removable
11. Exportable
12. Accessible
12a. Available
12b. Renderable
12c. Evidential
13. Redactable
1. Compliant
Organizations must comply with the legal and administrative requirements for recordkeeping within the jurisdictions in which they operate, and they must demonstrate awareness of best practices for the industry or business sector to which they belong and the business functions in which they are engaged.
1a. External recordkeeping requirements are known.
1a1. Laws of jurisdiction with authority over the record creating organizations are known.
1a2. Regulatory issuances of entities with administra-tive authority over the record creating organizations are known.
1a3. Best practices of recordkeeping established by professional and business organizations within the industry and business functions of the organization are known.
1b. Records created by organizational business trans-actions which are governed by external recordkeeping requirements are linked to an internal retention rule referencing the documented law, regulation, or statement of best practice.
1c. Laws, regulations, and statements of best practice with requirements for recordkeeping are tracked so that changes to them are reflected in updated internal recordkeeping instructions.
2. Responsible
Recordkeeping systems must have accurately documented policies, assigned responsibilities, and formal methodologies for their management.
2a. System policies and procedures are written and changes to them are maintained and current.
2b. A person or office is designated in writing as responsible for satisfying recordkeeping requirements in each system.
2c. System management methods are defined for all routine tasks.
2d. System management methods are defined for events in which the primary system fails.
3. Implemented
Recordkeeping systems must be employed at all times in the normal course of business.
3a. Business transactions are conducted only through the documented recordkeeping system and its documented exception procedures.
3b. No records can be created in the recordkeeping systems except through execution of a business transaction.
3c. Recordkeeping systems and/or documented exception procedures can be demonstrated to have been operating at all times.
4. Consistent
Recordkeeping systems must process information in a fashion that assures that the records they create are credible.
4a. Identical data processes permitted by the system must produce identical outcomes regardless of the conditions under which they are executed.
4b. Results of executing systems logic are demonstrable outside the system.
4c. All operational failures to execute instructions are reported by the system.
4d. In the event of system failures, processes under way are recovered and re-executed.
5. Comprehensive
Records must be created for all business transactions.
5a. Communications in the conduct of business between two people, between a person and a store of information available to others, and between a source of information and a person, all generate a record.
5b. Data interchanged within and between computers under the control of software employed in the conduct of business creates a record when the consequence of the data processing function is to modify records subsequently employed by people in the conduct of business.
6. Identifiable
Records must be bounded by linkage to a transaction which used all the data in the record and only that data.
6a. There exists a discrete record, representing the sum of all data associated with a business transaction.
6b. All data in the record belongs to the same transaction.
6c. Each record is uniquely identified.
7. Complete
Records must contain the content, structure, and context generated by the transaction they document.
7a. Accurate: The content of records must be quality controlled at input to ensure that information in the system correctly reflects what was communicated in the transaction.
7a1. Data capture practices and system functions ensure that source data is exactly replicated by system or corrected to reflect values established in system authority files.
7b. Understandable: The relationship between elements of information content must be represented in a way that supports their intended meaning.
7b1. Meaning conveyed by presentation of data are retained or represented.
7b2. System defined views or permissions are retained and the effects are reflected in the record represented.
7b3. Logical relations defined across physical records are retained or represented.
7b4. Software functionality invoked by data values in the content of the record are supported or represented.
7c. Meaningful: The contextual linkages of records must carry information necessary to understand correctly the transactions that created and used them.
7c1. The business rules for transactions, which minimally locate the transaction within a business function, are captured.
7c2. A representation of the source and time of the transaction which generated a record is captured.
7c3. Links between transactions which comprised a single logical business activity are captured.
8. Authorized
An authorized records creator must have originated all records.
8a. All records have creators which are documented.
8b. Records creators must have been authorized to engage in the business that generated the record.
9. Preserved
Records must continue to reflect content, structure, and context within any systems by which the records are retained over time.
9a. Inviolate: Records are protected from accidental or intended damage or destruction and from any modifi-cation.
9a1. No data within a record may be deleted, altered, or lost once the transaction which generated it has occurred.
9b. Coherent: The information content and structure of records must be retained in reconstructible relations.
9b1. If records are migrated to new software environments, content, structure, and context information must be linked to software functionality that preserves their executable connections or repre-sentations of their relations must enable humans to reconstruct the relations that pertained in the original software environment.
9b2. Logical record boundaries must be preserved regardless of physical representations.
9c. Auditable: Record context represents all processes in which records participated.
9c1. All uses of records are transactions.
9c2. Transactions which index, classify, schedule, file, view, copy, distribute, or move a record without altering it are documented by audit trails attached to the original record.
9c3. Transactions which execute a records disposition instruction, whether for retention or destruction, are documented by audit trails to the original record.
10. Removable
Records content and structure supporting the meaning of content must be deletable.
10a. Authority for deletion of record content and structure exists.
10b. Deletion transactions are documented as audit trails.
10c. Deletion transactions remove the content and structural information of records without removing audit trails reflecting context.
11. Exportable
It must be possible to transmit records to other systems without loss of information.
11a. Exporting protocols should be reversible.
11b. Functionality should be represented in a fashion that produces the same result in the target system as in the originating environment.
12. Accessible
It must be possible to output record content, structure, and context.
12a. Available: Records must be available.
12b. Renderable: Records must display, print, or be abstractly represented as they originally appeared at the time of creation and initial receipt.
12b1. The structure of data in a record must appear to subsequent users as it appeared to the recipient of the record in the original transaction or a human meaningful representation of that original rendering should accompany the presentation of the original context.
12c. Evidential: Record's representations must reflect the context of the creation and use of the records.
13. Redactable: Records must be masked when it is necessary to deliver censored copies and the version as released must be documented in a linked transaction.
13a. The release of redacted versions of a record is a discrete business transaction.
13b. The fact of the release of a redacted version of a record is an auditable use of the original record and therefore results in creation of an audit trail with a link to the transaction which released the redaction.