Thoughts on Criticisms to Make of the OFA’s Method for Analysing Marriage Patterns, as Adopted in STN’s RFD (the Little Shell PF and Burt Lake FD).
Steven Austin
Begun: November 12, 2006
In no particular order:
1. At the time the CTAG filed his appeal of the STN FD before the IBIA, the OFA’s research staff was divided and or uncertain as to what precedents had been set by previous OFA decisions. That the STN’s method was okay with the OFA, is supported by the fact that the OFA affirmed that analysis in its FD. Rather than having a consistent body of precedent, the OFA precedent on marriage patterns was completely incoherent, as seen in the notes, e-mails, and memos written by OFA researchers during the IBIA appeal process. Not only had the OFA’s researchers used different methods for calculating marriage patterns from one decision to another, they also used significantly different methods within the same decision (applying widely divergent time-frames – sometimes calculated every 20 years, every 10 years, or annually; whether the rate should be based on the longevity of the relationship or the initial year the marriage union was contracted; failed to use statistically random sample, rather doing the calculations based on capriciously choosing specific family lines (e.g., calculating marriage patterns within the family lines of tribal council members, rather than all of the petitioner’s members who married; or, calculating marriage patterns for unions that had produced offspring).
Rather than admit that they had used varying methods and that this was a problem, they took the position (with no support in logic or from the professional literature) that they could use STN’s method for evaluating marriiage patterns under 83.7(b)(1), were required (by past OFA precedent) to use the CTAG’s method for evaluating marriage patterns under 83.7(b)(2).
The OFA’s technical assistance and decisions are relied upon by petitioners to provide a model for their own work in putting together petitions for acknowledgment. When the STN’s method for calculating marriage patterns was reaffirmed by the FD, and the CTAG has not met its burden during the APA review (a preponderance of evidence) to show that the STN’s petition was in error (in fact, STN added to the body of evidence supporting its position, but the CTAG did not), STN has a reasonable expectation that the FD would be upheld.
The STN was put in the incredibly disadvantageous position of attacking an undeclared negative FD during the IBIA appeal process. Had STN had a full opportunity to appeal a negative FD, the research strategy would have been very different during the IBIA appeal process. For example, we would have looked at the literature on marriage patterns in a much fuller manner than that allowed by the ad hoc process created during the IBIA review.
Also, in Federal law the Agency is shown deference in its decisions when it has unique expertise in its area of administration. In this case, however, the OFA’s body of precedent is characterized by inconsistency (also acknowledged by Bob Jackson, OFA Historian), and the failure to observe accepted social science methodology, statistical standards, and basic logic. These characteristics make the OFA’s opus an unreliable basis for making any decisions. The inconsistency supports the STN’s position that the professional literature must be considered and the OFA open the issue up to comment from professionals in the field of social science statistics.
2. The OFA researchers were aware that various methods had been used in past OFA decisions (in fact, initially Rita Souther was quite ardent in her support of the method the STN and OFA both used for the FD, repeating the OFA’s received wisdom at that time indicating the marriage rates could legitimately be calculated either way, STN’s or the CTAG’s), asked questions about what the OFA’s precedents were (DeMarce and Souther seem solid that counting individuals was the way it had usually been done; Roth was less commital, particularly given his analysis in Little Shell PF, which adopted the CTAG’s method in 2000), and if there were any reasons they might want to go in a different direction than they had in the past; in the end, they chose to go with the one that seemed to least benefit the petitioner; more concerned about that (not appearing to be too favorable to the petitioner) than what the professional literature and statistical logic and methods required.
4. OFA weighed whether or not any past petitioners who had either been acknowledged or rejected for the same would have achieved a different result, depending on which marriage patterns method would be used. We did the same analysis, and concluded no one would have been acknowledged who might have benefitted by using the correct method.
5. OFA did not review the literature we submitted or the arguments that we made based upon that literature, with the exception of Robert Jackson (OFA Historian), who agreed that our method was the correct one. Simply attempted to cast doubt upon the literature review by stating broadly that the petitioner’s researchers did not state explicitly how many hours were spent conducting the literature review. A statement of how many hours were spent on the petitioner’s research has never been required. Rather, the OFA’s responsibility is to confirm or disconfirm the evidence and arguments in the petition materials.
The appropriate, professional response, and the one required by the regulations and the two supplementary “directives,” would have included a review of the literature as well as our arguments. In superficially dismissing the literature review, the OFA completely sidestepped its responsibility under the regulations and currently stated OFA policy and procedures.
Rather, than considering what the petitioner’s researchers had presented in the form of a literature review and professional arguments, the OFA appears to have decided that the method they wanted (or were ordered) to adopt was the CTAG’s method, and then they looked at earlier precedents to attemp to demonstrate that they had “always” done it this way (at least “when it mattered” under 83.7(b)(2)).
6. The OFA should not be allowed to flaunt accepted professional and statistical methods, whether their own recently adopted “precedent” (Little Shell, STN, and Burt Lake) supports them or not (and as we’ve established, their precedent does not consistently support their newly adopted method). The OFA researchers’ work is supposed to be governed by the principles of each of their professions (anthropology, history, and genealogy), not to the preferences or whims of the individuals.
7. The CTAG has taken the position that “marriage rate” and “rate of endogamy” are two different things. This is a misleading and false dichotomy, and the CTAG’s position is not supported by reason or the social science literature. In the context of the acknowledgment regulations at 83.7(b), there is only one marriage trend at issue: the change over time in the rate at which tribal members marry other tribal members (ceteris paribus, this rate is expected to decline over time, from first sustained contact to the present). It is a trend in the extent to which it characterizes the tribal membership’s overall behavior. The calculation that determines if there is a pattern of tribal members marrying other tribal members is the rate of endogamy. Specifically, to meet the requirements of 83.7(b)(2), the petitioner must present evidence that demonstrates there is a pattern of the petitioner’s members marrying each other at a rate of 50 percent or more. Thus, the marriage pattern is determined by the rate of endogamy.
8. STN’s position is supported by logic, statistical methodology, the regulations, and the guidelines for the regulations.
There is absolutely no conceivable justification for choosing to use the CTAG’s method for calculating endogamy.* It is not one of several legitimate ways that the calculation can be made. It is statistically flawed and must not be adopted.
*I can only think of two reasons for using the method called for by the CTAG. The first would be to compare the rate of endogamy for tribal members and non-tribal members. But to do that, one would need complete data for the marriages of non-Indians, which the CTAG does not have. Indeed, the STN did not collect data on non-Indian marriage patterns because the marriage patterns of non-Indians are not at issue under the acknowledgment regulations. Rather, it is the overall pattern of behavior of the tribal membership that is at issue.
A fictional example of this would be: Of 100 marriages for the whole town of Kent, Indians and non-Indians, for the decade 1800-1809.
The data for this fictional population would result in an endogamy rate of 75 percent (15/20=.75) for the STN membership and an endogamy rate of 88 percent (70/80=.875) endogamy for non-STN citizens. The percentage would not change if the calculations were made based on the number of individuals involved in the marriages:
STN endogamy rate 30/40= .75
Endogamy rate for non-STN members 140/160 = .875
This demonstrates that if the calculation is done using a valid statistic, the result will be the same, whether one calculates on the basis of marriages or individuals married.
The problem with choosing the CTAG’s method is that it lacks complete data on non-Indians, and erroneously makes its calculations in the following statistically invalid manner:
A total of 30 “marriages” for the STN tribal membership, and 50 percent of them are endogamous, rather than 75 percent.
The second way one could use a statistic similart to the CTAG’s would be limit the study to one sex or the other. But, one would have to predetermine whether one were interested in investigating only the behavior of males or females, in order to be consistent in one’s treatment. Then one would have a table, as follows:
This results is a 50 percent (10/20 = .5) endogamy rate for STN males.
But the CTAG does not propose this method. Rather, he is implicitly haphazard in whether he is considering the behavior of male or female tribal members. Even if he did choose to use this valid statistic, it would not comport with the requirement of the acknowledgment regulations, which call for a study of the whole tribe’s behavior with regard to marriage, which implies both male and female behavior. The OFA can offer no rationale for being implicitly haphazard in considering only male or female tribal members’ behavior, and they have not tried to do so. The OFA’s researchers also do not address the grounds on which they would choose to study one or the other (male or female), even though it is possible to do so.
Analogies and Comparisons
Make the comparison with the way the OFA calculates the 50 percent when evaluating residence patterns under 25 CFR 83.7(b)(2). If there are non-Indian spouses in the household, or non-Indian step-siblings or half-siblings, they are not included in the count. The OFA should do the marriage patterns calcuations the same exact way, by eliminating the non-Indians from the count.
This is not about the wording of the regulations, the guidelines, or previous decisions of the OFA. It is about choosing to use a statistically valid measure for understanding the behavior of the Tribe’s members. The use of the invalid statistic proposed by the CTAG, and currently accepted by the OFA, must be discontinued. It has not caused harm to any petitioners thus far, with the exception of STN.
Choosing the CTAG’s invalid statistic results in the following irrational result. Take an example of two STN tribal members who are married to each other in an endogamous marriage from 1800 to 1810. The CTAG would count that as one marriage. If these same two individuals divorced in 1810 and remarried in 1812, this time they each marry non-Indians, this would count as two marriages, according to the CTAG’s invalid measure.
Current status of the literature review.
The statistic, even when done correctly is conservative, underestimating the tendency of tribal members to marry each other. This is because of the great imbalance between the number of citizens in the Indian community and the number in the non-Indian community, the latter being much smaller than the latter, as well as a skewed sex-ratio (which I pointed out was the case for STN in one of my earlier papers in 2003).