About our database
Confronted by systemic disinterest by Costa Rica’s Public Defender’s Office, a group of journalists and attorneys decided to voluntarily build a database to measure the performance of public and private defense attorneys. The database took four years to complete and was based on thousands of judicial rulings.
Our goal was to fill the gap in information left by the Public Defender’s Office, which despite its annual budget of $56 million, appears uninterested in filling it.
This database is the first of its kind. It is the largest effort to date in Costa Rica to compare the performances of public and private attorneys. After several rounds of filtration, analysis and other applied testing (detailed below), the primary tendencies that emerged did not vary, making the database a valid source of information.
Of course, inconsistencies can be found because the original source of all of the information (judicial rulings) isn’t always precise. Second, despite putting controls in place, no database this large is error-free, due to human error (8,000 rulings and 11,000 offenses were analyzed). Third, sufficient official data is lacking that would have allowed some of the variables to be cross-checked.
We are aware that our work is perfectible, so we are offering complete access to the database that we built. This allows readers to review it and, if necessary, alert us to inaccuracies or errors.
We will evaluate all corrections and determine if they change the results of our analysis. As with all proper use of scientific method, we believe that new information will allow us to reach updated and better conclusions.
Corrections and comments can be sent to: [email protected]
Transparency and openness
We decided to publish our study knowing that the database was perfectible for three reasons:
1. There is no indication that the potential errors are extensive and would cause the model’s primary results to drastically change.
2. The Costa Rican Public Defender’s Office has taken a systemic disinterest in generating statistics that would help to evaluate the performance of its attorneys. This would allow them to be compared with private defense attorneys, whose clients tend to be wealthier. Additionally, the country’s judicial branch currently cannot issue rulings based on complete and precise information. Beyond these two factors, there is a need for citizens to understand and discuss relevant issues. Apathy in the management of public institutions is dangerous. Not only does it hinder self-evaluation and criticism by the seats of governmental power, particularly given the lack of basic statistics, but it also allows these powers to benefit. As such, private, voluntary efforts to inspect in good faith the labors of the state, such as what Univision Noticias seeks to do with this investigation, are blocked.
3. Journalism should aspire to a high degree of certitude for stories that are published. This report is no exception. But our study uses social research methods, and in the social sciences it’s usually impossible to obtain absolute certainty; more so when encountering complex research problems. Often, the generation of knowledge and public debate in a certain area leads to conclusions that are based on the best information available at a specific moment.
Details regarding the filtering process
Building this database was a four-year effort. In the last phase, with the of the State of the Nation, we managed to finish the filtration of the database by using digital case file data possessed by the judicial branch.
In this phase of filtering, several risks were identified regarding the database’s consistency:
The data included in rulings often lacked standardization, which produced many empty cells. One of the missing variables in some rulings was the type of defense attorney. As important as it is, the judicial branch is still unaccustomed to ing this detail.
Electronic rulings that are a primary source of data have proven to contain incorrect information on occasion.
Building such an extensive database over a long period of time, and with a group of people working simultaneously, generates human typing errors. To minimize risks, the team conducted a series of cross-checks last year with both internal judicial branch sources and external ones. This filtration concentrated on finalizing rows that had the variables “type of ruling” and “type of process.”
Some of these applied controls are:
1. The classification was cross-checked with a list of public defenders.
2. The status of the attorney was checked at the Attorneys Association.
3. A database (report) generated by the judicial branch’s official statistical system (SIGMA) was requested, from which some rulings were verified to fill out the information. The report’s usefulness, however, is limited because the system has gaps and is only available for recent years, not the entire decade analyzed.
4. It would have been possible to recover even more information by reviewing physical case files stored at the court, but access to these files was denied. The denial was based on a Superior Council resolution (Session 76-16, on Aug. 11, 2016, using the argument of the coordinating judge).
5. In cases of doubt, where a defense attorney was classified as both public and private (listed as serving as both during the 10 years analyzed), this variable was cross-checked. It was done by calling many of them on the telephone; reviewing rulings from other courts (especially the Penal Branch of the Supreme Court) to corroborate their status around the time the Criminal Court in Goicochea was intervened; and through Internet searches. As a result, errors were corrected in which the ruling did not coincide with the database, or the ruling contained incorrect data (220 cases were corrected, which represents a little more than 3% of the database).
6. In cases where doubt persisted, and the attorney couldn’t be located in other sources, the decision was made to eliminate the offense. After this filtration, the base used for the model included 6,584 offenses.
7. Analysis models were calculated before and after filtration, maintaining general research tendencies in both cases. The statistical team from the State of the Nation Program also carried out a study of 100 routines in which the elimination of approximately 680 cases was simulated (by not being taken into consideration). The model’s central tendencies remained intact.
