Court Data: Now You See It, Now You Don’t

Some time ago, over lunch, I spoke with a justice who leads a team at a supreme court. Our conversation began with UNODC’s work to support women judges, but soon turned to a deceptively simple question: how should courts use performance data?

This question brought back a vivid memory from my own time as a judge. As an experiment, our team received a printout showing how many judgments each judge had produced over a given period. One colleague appeared to be performing exceptionally poorly. Yet the figures did not lead to a constructive conversation. My team leader felt he could hardly send this distinguished former lawyer to a judgment-writing course without embarrassing him. The printouts disappeared—and we never saw them again.

From embarrassment to improvement

The problem was not the data itself. It was the culture surrounding it. The figures were treated as a ranking—as evidence of winners and losers—rather than as a starting point for learning. So, I asked my colleague how she used the information generated by her court’s case-management system. Did it help her identify difficulties? Did she discuss it regularly with team members? Could judges see their own results, or those of the team as a whole?

She told me that she used the data primarily to spot problems. That is valuable—but it is only the beginning. I believe the entire team should have access to the team’s performance data, provided the figures are interpreted carefully and used fairly. Transparency allows judges to understand the average, see patterns, identify colleagues who may be able to offer advice, and recognize where they themselves can help. Used well, shared data can strengthen a culture of cooperation rather than competition.

What court data can reveal

Three of the most common, useful measures concern timeliness: clearance rate, time to disposition, and the age of the active pending caseload. Together, they show whether a court is keeping pace with incoming work, resolving cases within expected time frames, and allowing unresolved matters to grow old.

These measures are useful precisely because they can expose problems that are not obvious from individual output alone. While helping to design a digital procedure for appellate courts, for example, I discovered that the courts’ biggest constraint was a shortage of hearing rooms. If a case must wait nine months for a room, improvements elsewhere in case management will have only a limited effect.

Court data can reveal much more. Changes in the volume or type of filings may show whether diversion programs, alternative dispute resolution, or procedural reforms are working. Backlogs—cases that should already have been resolved—may point to bottlenecks, staffing shortages, or mismatched funding. Those pressures can also fall disproportionately on particular groups of court users. Data should therefore prompt questions, not merely produce rankings.

When measurement becomes a target

Performance measures also create risks. When funding or prestige is tied too closely to a metric, people may be tempted to manipulate the metric rather than improve the underlying work. In one recent case, a former colleague faced criminal charges for allegedly falsifying the signatures of two other judges on decisions he had made alone. The court received more funding for cases decided by a three-judge chamber than for cases decided by a single judge. His defense was that he had acted in the court’s interest.

That example captures the paradox of performance data. Hide the figures, and courts lose opportunities to learn. Turn them into crude targets, and the figures can distort behavior. The better approach is transparent, contextual, and developmental: share the data, discuss what lies behind them, and use them to improve systems as well as individual practice. Court data should be a mirror, not a scoreboard—and never a reward for making the numbers look good.

 

A Writer Needs Readers

Someone recently told me they had read my work — and appreciated it. That simple moment mattered more than I expected. Hearing that my long commitment to IT for courts still resonates reminded me that I still have things to say, and that there may still be people who want to hear it.

Over the past months, I’ve been rethinking the role of technology in justice. Technology evolves, court users’ needs shift, and the principles that anchor justice — impartial judging, independence, fairness, transparency, reasonable time — must continue to guide how we design and govern digital systems. As I wrote, this means ensuring that IT, including AI, supports performance, not just efficiency.

What surprised me was discovering that this “new thinking” wasn’t new at all. It has been present in my work since the early 2000s, from my first article in Nederlands Juristenblad to Doing Justice with Information Technology and even earlier, in a 2002 speech on “IT dreaming and reality.” The thread has always been the same: technology should strengthen the core values of justice.

My fascination with IT began even earlier. In 1984, during an internship, I encountered two enormous Wordplex machines used to produce arbitral decisions. It was primitive by today’s standards, but transformative then — a glimpse of how digital tools could improve the work of justice. That spark eventually led to the first information policy plan for the Netherlands judiciary in 2000, framing IT as a driver of performance improvement.

Rediscovering this history has reminded me why I started writing about justice and technology in the first place — and why I want to continue. There is still experience to share, still lessons worth passing on, and still readers who care about how courts evolve.

So, I’ll keep writing. And I hope you’ll keep reading.

 

How many ways are there to approach court reform? The #JARA conference in Limoges, France 10-12 June, explores a wide range:  #courts and #judiciaries are no longer taken for granted, the cliché says. How to approach and research court reform to increase legitimacy, trust and accountability of courts? Think independence, governance, managerialisation, procedural justice, trust, confidence and accountability and judiciary professions.

Information technology plays a major role in court reform. #IT must support fair #justice, not cost reduction. This poses issues for judicial independence, such as judicial digital governance, illustrated by examples of E-justice concrete cases and justice data spaces. Artificial intelligence #AI poses new issues for reforming judicial processes, also illustrated by concrete cases.

https://jar-association.eu/getting-ready-for-limoges-2025/

Twenty years ago, I  was on the Justice millennium task force.  We  were tasked with making scenarios for all sorts of eventualities in case of the millennium bug causing various forms of computer failure. For the under-30s: this bug could not recognize 2000 as a valid year and might stop our computers from working properly. We ruminated on eventualities: what if the electricity stops? If the electricity stops, what about the technology that manages the water table? A serious issue in a country that is largely under sea level. What branch of justice should be given priority when this happens? We quickly realized how futile this discussion was. When everything is two feet under water, giving priority to the family courts or even the juvenile courts was no longer really an option.

So, twenty years later, how do courts cope with the emergency of COVID-19? Most courthouses were closed, except for emergencies. Family cases might not count as emergencies any time soon. But what if, as happened in the Netherlands, a father cannot exercise his visiting rights because of COVID-19? My guess is that resolving this would require a face to face hearing with both parents. Besides, courts are needed as a check on governments’ use of their emergency powers, too.

Courts are turning to information technology to keep their business open. Staff and judges may be able to work from home. Some courts allow filing documents by (more or less secure) email to keep contaminated paper out of the courts. Various videoconferencing tools are in use to for hearings when necessary. Courts find them surprisingly easy to use. The hardest problem to solve is public scrutiny. Both the Federal courts in the US and the Dutch courts are allowing repeat users, such as the press, to follow hearings by video. The Dutch Bar Association urged the judiciary to open up more possibilities for work to continue. Online tools are becoming the rule, and the case judges have discretion to order face to face hearings where absolutely necessary. A second category of (somewhat less) urgent cases has been identified.  

How are courts using online tools to cope with the COVID-19 pandemic? On Sunday, March 29, I asked my Twitter and LinkedIn contacts for information about online tools courts use during the COVID-19 pandemic. Thank you all for sending sources of information. I now have some information on 90 geographic entities, countries, states and territories. They come from the International Association of Judges – thank you, Paulo Roberto Dornelles Junior, Labor Judge Member of the International Relations Secretariat of the Brazilian Magistrates Association for your March 22, 2020 report. And thank you, Norman Meyer, for pointing me to the excellent NCSC web page, by the National Center for State Courts in the US. My third major source is remotecourts.org.

Here are some first impressions. Clearly, the pandemic has not hit each of those entities to the same extent, and this also influences the measures taken. 9 entities appear not to have taken any measures yet: some states in the Mid-West of the US, some in Africa. Some court systems stay open and leave decisions on how to proceed to the case judges, taking the local COVID-19 regime into account. Most courts have cancelled in-person hearings. A number of systems have closed their courts altogether, except for urgent matters. They postpone cases until after the crisis where possible.

My main interest is in the use of IT to continue providing services. I looked at remote work for court staff and judges, remote hearings, and digital filing. Remote work for staff and judges is mentioned by 5 entities. My guess is that quite a few entities do not think this is worth mentioning, since it is business as usual. 13 entities say they use remote hearings, mostly for urgent cases only. They mention Skype for Business, Microsoft Teams, Zoom video and the Vidyo app as their technologies of choice. 6 entities mention digital filing, which may include using dedicated email. Here again, where digital filing is not exceptional, why mention it?

So, it seems courts and courts systems where working remotely is business as usual have few problems staying open for business. Others now seem to be scrambling to catch up. I need more information to verify these first impressions, so keep the info coming in, either at my Twitter account @doryontour or my linkedin page. 

International women’s (judges?) day

Someone recently suggested instituting an International Women Judges Day. Meanwhile, I guess we had better make good use of International Women’s day to pay some attention to women judges. Having been a woman judge for more than thirty years, you can bet I have stories to tell, some of them even amusing. But that would be misusing this space.

In the Netherlands, the first woman judge was appointed in 1947. A hundred years of debate had preceded her appointment. Those opposed (men) argued that women were much too emotional for the task of judge. Today, this seems almost amusing, but at the time it must have been painful for all those women who yearned to put their talents to good use. In the 1970, my civil procedure professor argued that women make better judges than men, because they are less confrontational, less threatening and more conciliatory.
Now that women judges have a numerical majority in the Dutch courts, two of my students at the Amsterdam University of Applied Sciences wanted to research the matter of women judges’ emotionality. They interviewed (1) male convicts about being convicted by a woman judge, (2) judges and lawyers about differences in emotionality among judges, and (3) lawyers and judges about appealing to women judges’ sentiment in the courtroom.

Almost none of the convicted men could recall whether they had been judged by a man or a woman, and almost none of them objected to being judged by a woman, except those from Moroccan descent. Some of their lawyers guessed this may be because they appeared to have problems with authority generally.

A prominent politician claimed that, since there were so many women judges, sanctions for violent crime had gone down. Both judges and lawyers, irrespective of their gender, agreed there was no noticeable difference in emotionality between women and men judges. Nor did they notice any difference in the sanctions. After all, there are guidelines for sanctions to be applied for most crimes.

A prominent male lawyer had boasted to the press that he could soften women judges with his suave tone of voice, the blink of an eye or a joke. The judges, when interviewed about this by my students, considered this unprofessional, exaggerated, very wrong, downright silly, ineffective, and an overestimation of his personal charm.

This may close the debate on women judges’ emotionality, but a new discussion is on the rise. Is the Netherlands judiciary becoming too “feminine”? Selection profiles for judges include, besides legal skills like argumentation and deciding disputes, competency in strong analytical skills, listening, persuasion, organization, cooperation, self-confidence, authenticity and flexibility capacity to learn, reflect and decide. I do not think those are specifically feminine skills. And is the judiciary losing some of its authority? Surveys show that confidence in the judiciary in the Netherlands has been going steadily up in the past years.

Judicial Integrity, IT and AI in Doha, Qatar 

On 25 and 26  February 2020, the Global Judicial Integrity Network held its second High Level Meeting, in Doha, Qatar. The overarching topic is always increasing judicial integrity, but this time the two big themes were gender diversity and information technology. My contribution consisted of a plenary panel discussion on information technology and artificial intelligence. Here is a summary of my contribution to the discussion.

Left to right: prof. Karen Yeung University of Birmingham, judge Madan Lokur from India, panel chair Roberta Solis, Judicial Integrity  team leader UNODC, myself, Judge Victor Momotov of the Russian Federation, Judge Ju Yeon Lee, from the Republic of Korea. (thanks to Judge Kees Sterk for the picture)

Roberta Solis sent us all questions beforehand.

My first question was on experience with in house IT development in the Netherlands.

Like most other judiciaries, the NL judiciary uses many different kinds of information technology.

Case management, office technology, multiple web sites with information, news and published case law (50.000+ judgments annually), court management tools, an intranet, email, and some e-filing and digital procedures. All this is managed by the judiciary’s own IT organization, under the Council for the Judiciary of the Netherlands.

An important reason for choosing in house development is continuity in knowledge: keeping the knowledge that is developed inside the judiciary. But there is a more important reason that is not always recognized. IT has reached a level of development that includes all work processes. It is no longer a tool, it is becoming an environment. And since it includes the primary, judicial work processes, the choice was to do that under judicial control.

The way to achieve this was to mix in house expertise with individually contracted specialist experts. The methodology of choice was agile product development, with judges as product owners. This ensured strong judicial involvement in the software development.

Some lessons to take away from this experience so far:

Development is not the most difficult issue. The most difficult issue is governance: changing work processes and implementing digital procedures require strong decision making processes geared for innovation. Most judiciaries are geared to be production organizations: they process cases. For going digital, this governance setup needs to change.

There are also some options for innovation to be gleaned from the experience in the Netherlands. The most important variables are the need for changes in the regulation and the length of the procedure to be digitalized.  

1 simple, short term processes can be replaced in one operation if they do not require changing regulations

2 more complex, longer running processes should be changed step by step, especially if they also require changes in regulation

3 setting up an entirely new process avoids the risks involved in option 2

4 setting up an entirely new institution creates the possibility to design approaches that leverage the power of going digital.

Roberta’s second question for me was about the CEPEJ charter for ethical use of artificial intelligence (AI)  in the administration of justice.

CEPEJ is the Council of Europe’s Commission for the Efficiency of Justice. CEPEJ decided to draw up the Ethical Principles in order to ensure responsible use of artificial intelligence in compliance with the European Convention on Human Rights and the Convention on the Protection of Personal Data, to help improve the predictability of the application of the law and the consistency of court decisions and to prevent discrimination.

In June 2018, CEPEJ held a session with experts on the workings of courts, legal experts and experts on the anthropological and  philosophical aspects of the emergence of artificial intelligence. CEPEJ also conducted a survey of the state of use of AI in the judiciaries, which showed that that use is still very low.

Experts produced reports on operating characteristics, legal reasoning, explaining judges’ behavior in retrospect, data protection, and predictive justice tools, that are included in the final reports as annexes.

The 5 principles in the final document are:

1         Respect for fundamental human rights

2         Non-discrimination

3         Quality and security

4         Transparency, impartiality and fairness

5         “under user control”

The Ethical Principles, therefore, also pose some new challenges for judiciaries.  

The Principles need to be operationalized, in regulation as well as organization. This is going to be a big challenge for judiciaries, as they will need to safeguard the integrity of their own data and their information systems. They also need to make their information machine readable in order for AI to make more effective use of it.  

CTC2019 Day Three

On the final day of CTC2019, there was still some interesting stuff to take with us. I attended a panel discussion about introducing Online Dispute Resolution in court systems. What makes this so interesting, is that it does not replace an existing court process to improve it, but it is an entirely new process, designed to support self-representing litigants in representing themselves in a process that is as simple as possible. In Utah, they “have made a wild animal”. I like that, because experience shows that replacing existing court procedures with a digital version may not solve any problems. However, designing an entirely new procedure leveraging the power of IT can increase access to justice and reduce disposition time. Initiatives also exist in Alaska and Hawaii. In Alaska, the procedure will be for traffic tickets, which poses a specific set of problems because it is a criminal procedure. For Hawaii, still in the development phase as well, the big benefit is in handling your case online instead of having to travel to the island where the court is. Utah is piloting its wild animal, and I will follow it to see where it goes next.

CTC2019 Day Two

Day Two kicked off with a talk on cyber predictions for the 2020s by Dan Lohrman, Chief Security Officer at Security Mentor Inc. Clearly, cybersecurity is increasingly becoming an issue for courts, and for everyone else for that matter. A major factor here is the fact that we have come to depend on technology for more and more purposes. Our smartphones are our tool to organize our lives: banking, smart homes, online ordering. Data security is a major issue for courts as they go increasingly digital in their work processes. Did you know that less than half of all smartphone users use passwords or biometrics to protect their data?
The rest of the morning was devoted to Tyler Technologies and some other vendors presenting their Showcase Solutions, mostly case management systems.
In the afternoon, the presentation by David Robinson, visiting scientist to the AI Policy and Practice Initiative, on Civil Rights and Pretrial Risk assessments, was an great follow-on to our workshop on AI on Day One. Background to this presentation is the effort to reduce pretrial detention: getting rid of bail, and detention only for those situations where is it really necessary. Research shows that measures to reduce pretrial detention often result in increased detention, for instance in the case of the New Jersey Initial Release Decisions. Pretrial Risk Assessments cannot safely be assumed to reduce pretrial incarceration, and it may even displace onther reforms that would be effective. For instance: narrow who should be jailed, robust hearings soon after arrest  - in the Netherlands, an arrested person needs to be seen by a judge within 48 hours after arrest - new supportive services to mitigate the risks - a simple thing is to text message defendants reminding them of their hearing - and get rid of the bail system as a routine measure. WHen using the assessment, ensure transparency, community oversight and governance, ensure decisions are rare, deliberate, and independent of risk assessment.

CTC2019
On Day One, the conference was kicked off with a keynote by prof. Richard Susskind. He gave us a preview of his new book that is coming out in November: Online courts and the future of justice. Traditional courts will lose respect, unless they go online, he says. Ten years ago in Denver, we talked about the growing numbers of self-representing litigants and how they could be helped to get justice. These days, they are somewhat harshly called “SRLs”, but for the rest there is also good news. Some court systems are developing or piloting online dispute resolution, aka ODR, tacked on to the front end of their court procedures. Justice Himonas of the Utah Supreme Court tells me their ODR - still in a pilot in a few courts - has resulted in more defendant engagement, which was also the intention with the project. Other states like Alaska and New Jersey are doing similar things. In the Utah courts, parties in a small claims case have the first 48 hours to negotiate among themselves, and then a facilitator steps in to guide them through a mediation-type process. If unsuccessful, they can opt for a simple court procedure or a traditional one. If the evaluations are positive, it will be rolled out to the entire state of Utah. The mediation phase is mandatory, but so far no one has used the opt-out possibility. The whole process is online.
And yes, also on day one, I moderated a workshop on Artificial Intelligence, with prof. Nicolas Vermeys of Montreal University, the Cyberjustice Lab, and Shannon Salter, chair of the Civil Resolution Tribunal in Vancouver. What is AI, how does it work, how is it applied in Vancouver, what are the known drawbacks, can they be countered with the Ethical Principles developed by the Quality Group of the European Commission for the Efficiency of Justice.

Reflecting the past year

Beginning of August 2019, I uploaded four new articles to my publications page. They are a reflection on what happened during the past year. The first article, called De legal tech van de rechtspraak (the courts' legal tech), in Dutch, was a contribution to the Legal Tech special of Computerrecht, published in April 2018.  I could not upload it here sooner because of restrictions imposed by the publisher. It is a description of the state of court IT in the Netherlands judiciary in the spring of 2018. However, in June 2018, the Judicial Council decided not to implement the digital procedure for commercial claims. The Council also decided to discontinue the Quality and Innovation (Q&I) Program. This development left me quite speechless for a while. But when, in November 2018, The Tijdschrift voor Internetrecht asked me to reflect on the lessons from the Q&I program, I produced another article in Dutch, Lessen leren uit KEI (Learning lessons from Q&I) published in January 2019. Meanwhile, I had acted as a speaker at a number of conferences on Court IT. I generally held about the same presentation on court IT and its pitfalls. I summarized the presentation in an article in English: Digital justice, nice to have but hard to achieve. I publish an early draft of it here, since its official publication is still pending. This article presents both the legal tech from the first article and the lessons from the second one, as well as some conclusions on ways forward for court IT. On a more positive note, I took part in the ongoing discourse on the ethics of artificial intelligence. This resulted in a contribution to a special edition of Les Cahiers de la Justice, on Les Défis de la justice numerique (The challenges of digital justice). This article, Quelle place pour l'intelligence artificielle dans le processus de décision d'un juge (the place of artificial intelligence in the judicial decision process), is the French version of my earlier paper for the Netherlands Parliament, also on the publications page.

Court digitalization: not easy!

A visit to the showcase court of a judiciary somewhere in Europe. The court clerk shows us how he works digitally. If he wants to view a document in a case, he looks up the case in the registration system, notes the case number on a piece of paper, closes the registration system, opens the file viewer, taps in the case number, and sees the documents in the relevant case. The PDF documents in the file are arranged in chronological order, that is, if they are properly organized. The secretary checks new arrivals, completes the registration system where necessary, and allows cases into the system.

The courts in this country went paperless a few years ago. Everyone can submit a case digitally in the portal. What happens behind that portal is a different story. Some adjustments were made to the old systems that date back to the 1980s. The users, judges, and court clerks, are not happy. They have more work, and the courts also use more paper than before. After all, they now have to print documents themselves if they want something on paper, because they no longer receive their information on paper.

Now that I am retired, I have time to look around in Europe to see how the courts are working digitally. When I give a lecture or a workshop somewhere, I also pay a visit to the people who develop the IT for the courts. My impression: they all have the same struggle: How to get from a traditional, paper-based work process to one with only digital information. Some do it step by step; maybe that works, but it takes a long time. Others start with a digital case file. That also takes a lot of time. My showcase country started at the front end, and that now gives all sorts of problems with the work processes within the courts. Others have a wide-ranging program with a lot of funding, and whether that actually works is uncertain. In all cases, the transition places many harsh demands on the judicial organizations. The system and the process legislation will quickly become so inextricably linked that they can no longer be seen as separate. The work processes must be standardized and arranged from the outset. Decision-making processes must be in place that determine who is allowed to decide on the work processes and the system and when. Experience shows putting user friendliness first is key, and making savings should not be the overriding goal. That makes for dissatisfied users. After the experience with the digital commercial claims procedure in the Quality and Innovation Program (more about that some other time), my impression is that the judges in the Netherlands would never have accepted what the judges in my example judiciary have to put up with.

Another consequence of retirement: this was my last blog for Dutch legal magazine Mr-online. This is the last time that I, as usual just before the deadline, send my blog to Mr-online. Not because I have nothing more to say, on the contrary. My agenda is full of guest lectures, conferences, panels, fora and consultancy assignments. There are interesting developments, such as the discussion about artificial intelligence and what that can mean for the judiciary. I will keep writing about it on this, my own Technology for Justice blog. I report new blogs to my followers on LinkedIn, Facebook and Twitter.

It is time to make room. Mr-Online, thank you for having been my forum for almost ten years.

Artificial Intelligence

In March 2018, the Justice and Security Commission of the Netherlands Parliament Second Chamber held a round table on artificial intelligence in the justice domain. Here is the summary of my speaking notes, on AI and courts. The original was, of course, in Dutch. This English version I wrote for the session on Artificial Intelligence by the Commission for the Efficiency of Justice (CEPEJ) session on June 27 2018 in Strasbourg.

AI for courts, in brief (this is the summary)
What use can artificial intelligence (AI) have for courts, and what does that take? In court cases, judges reduce complexity, but all court work is by no means complex, bespoke work. Courts do not process all cases in the same way, and consequently, they need information technology suited to the different ways. Therefore, AI can be useful for different types of courts cases in different ways. Some forms of AI have already proven themselves in practice. But will robots replace judges, as some people have been claiming for more than twenty years? There is still no evidence to support it. Article 6 of the European Convention on Human Rights prescribes fair procedure. It will take a lot of work to make AI conform to that standard. Legal information needs to be structured and given meaning, in order to make the information not only readable, but also actionable, including decision making. Providing an explanation is, in the foreseeable future, not yet feasible for AI. AI can help people looking for information, parties in a case and judges with structuring information, and if legal information is enriched, also with advice and suggestions.  read the full note here

Court IT: we must, we can, but it’s not easy

A while ago, a judicial colleague published a column in one of the Dutch online papers. My colleague pointed to rising cost and longer duration of the digitalization of the courts. The digitalization was done in the Quality and Innovation Program. In Dutch the program is called Kwaliteit en Innovatie (KEI). The courts are, apparently, held ransom by the vendors. I reacted to his column in my monthly blog for one of the Dutch legal magazines. My blog went viral. Reason enough to give it some wider circulation by including it in here, in my Technology for Justice blog. For those of you who read Dutch: here is the original.

My question: if digitalizing turns out to be more difficult than expected, should we give it up?

Dear colleague,

Should I react to your column entitled “IT-vendors hold judiciary in a tight grip”? I wondered if I should.  Then, I read an earlier interview in which you wanted to abolish the redrawing of the judicial map, as well as the Council for the Judiciary. Fortunately, we judges enjoy freedom of speech. I am not going to debate the merits of your viewpoints. I had already decided not to publish this blog. But then, my IT team asked to please publish after all. The negative publicity made them very uncomfortable. So, therefore, I use this opportunity to list a few facts.

The parliamentary hearings on government IT-projects have made clear that an IT project can be considered a failure when the software that was developed is not implemented. Delay of rising costs are, in themselves, not a reason to consider a project a failure.

The first KEI-systems went into operation in 2015. Since April 2015, the system handled more than 20.000 asylum cases. Users in the courts are very happy with the system. Hearing planning is now a matter of hours instead of weeks. Since November 2015, bankruptcy supervisors can communicate with the courts in all types of bankruptcies. Well over 80% of bankruptcies are now handled this way. Professional supervisors have started to communicate digitally with all the courts in November 2017. Since September 2016, more than 650 (over 900 by mid-March, DR) commercial claims were filed with the new system, in two pilot courts. A few during a voluntary test phase, but most in the current compulsory filing pilot phase. Some cases have already been completed. One case, with a full hearing, was concluded within 7 weeks from filing. Users are not entirely happy with the system yet, and therefore the system has not been rolled out to all courts. That will happen later in 2018. So, the KEI software has been in use for a couple of years. 

The Council of State, in its 2014 advisory on the digitalization legislation, pointed out the risk of innovating procedure and digitalizing in a single operation. The Council also pointed out that earlier suggestions to drop the distinction between claims and requests and handle all cases on the basis of a request, were not followed, and that this complicates the digitalization.  De legislator did not follow this advisory. This has made designing the civil digital procedure more complex, hence more expensive than expected. There were a few other things that could not be foreseen in 2014. For example, article 113 of the Code of Civil Procedure that was added in a later phase of the legislative process. This article upended the design that we had already made and built, and sent us back to the drawing board. As a consequence,  we could only give lawyers access to the new systems, and no one else.

Digitalization needs to be kept simple. That is not a new insight. The Netherlands Accounting Chamber investigated government IT projects at the request of Parliament. The Chamber reported extensively in 2007. The dynamic of politics makes government IT-projects complex. Look at the Tax Office. That took a long time, but by now we all file our taxes on line.

Suppose: we know we have to keep things simple, but from the Accounting Chamber’s reports it is clear that complications will arise anyway. What do we do? Forget about digitalization? I cannot speak for the legislator or the Council for the Judiciary, but I can speak for myself, as the product owner of the civil procedure. I felt we should go ahead, and seize this opportunity that might not arise again for the next ten years. With every decision, we tried to apply the simplicity rule: can we make it simpler, is it necessary at all? It turned out we could not always do that. Legislation, the environment of bailiffs and lawyers, requirements and wishes from the users in the courts, technology itself, security needs, and a host of other factors kept us from applying the simplicity rule consistently. But some procedures have been digitalized, and they work. Civil procedural legislation changed so much that the courts need to put a lot of effort into implementation, and that is another reason why the new system has not been rolled out to all courts yet.

Dear colleague, we judges are the guardians of the existing legal order. Our work is looking back and deciding who need to get the blame for what went wrong. There is nothing wrong with that. But looking ahead and envisioning how to innovate does not come naturally to us. So, court innovation is difficult. If we are afraid to be blamed if something goes wrong, we will avoid innovation. Blaming in retrospect will not help innovation. But it must be done, because we are losing our market share to digital alternatives, and we do not do enough to give ordinary people access to justice.

CTC2017 Day 3

On Day 3, I attended prof. Fred Lederer's wide-ranging and very interesting talk on legal aspects of Artificial intelligence and the Internet of Things. This summary of what I think were the highlights will not do it justice.

As algorithms are becoming such a determining factor, there was a suggestion to treat algorithms like corporations. 

The essence of data analytics is that we will find things we did not know were there. 

Good source for the Internet of Things: article by  Maciej Kranz in Harvard Business Review. 

Will existing law answer issues posed by the new technology? What is the "right" answer for the society involved?

What if the police want to pull over an autonomous self- driving car for speeding, while the human inside the car is not driving it? As a judge - professors are good at framing questioins, but judges need to answer them - my first question is: can a self-driving car speed? Or is it designed to obey all traffic rules including speed limits? I would say yes. 

And what about someone who hacks a self-driving car in order to cause an accident with it? Again, there is an unanswered quesstion behind this one: what constitutes the crime? Hacking? Causing the accident?

Pretrial detention algorithms are being used by judges more and more. It is also found to be problematic, for instance biased against African American defendants. My observation is that we know very little about the causality of the factors involved in recidivism. 

In the conference's Endnote, all big themes were addressed: 

Cybersecurity

Innovation

What comes after digitization?

Artificial intelligence

Designing for the user

User testing for innovation - jury service on their phone

Translation technology

Agility, allowing for failure

Organizational ITmaturity

Next generatioin apps 

Cloud computing and surviving thehurricane. In one of the courts, justice never stopped in spite of the storm. 

New data uses - e-notifcations reduced Failure to Appear by 50-20%.

CTC2017 Day 2

Fro me, Day 2 was dedicated to innovative stuff for access to justice.

MJ Cartwright showed me a phone app that helps people deal with traffic violations, including the trial if it needs to come to that. Over lunch, I listened to the problems of a CIO from the Midwest, who felt there was so much that had to be done, although their systems were much better than people expected of the Midwest. And I had a wonderful conversation with Margaret Hagan, director of the Design Lab of the Stanford Law School. More about Margaret later. 

In the afternoon, I participated in a workshop on improving the user experience - lessons from law schools and innovation labs. 

NULawLab's Dan Jackson explained their approach. They developed RePresent, a game to teach people how to represent themselves. Their latest product is a game called Angry Tenants. 

Margaret Hagan introduced the 4 core strategies for improving the user experience: 

Customizing

Conversational Artificial intelligence

Visual models

Coordinating services in the local community

An example of an AI tool: you talk about your crummy landlord on Twitter, and our AI can then send you an advice to go to your local service center for advice. Would that work? Or does that go too far?

Visual models - how effective are they? Different visual models work for different people. 

Christopher Griffin A2J Lab Harvard explained how they do randomized trial testing to test for effectiveness, for instance of tools like self-help materials, court user notices. In the case of court user notices, the default rate dropped by half after the text had been improved.

Another example was triageing lawyers for cases where they are going to make a difference. 

The rest of the afternoon we spent doing a design sprint in 1 hour. Usually, design sprints will take longer than that. It was a fun exercise, and people came up with very innovative solutions. 

CTC2017 day one

 CTC2017 is under way in Salt Lake City, Utah. This is my 10th Court Technology Conference. What will I take away from this one?

The opening keynote, by Mark Lanterman, Chief Technology Officer at Computer Forensic Services, took us on live a tour of the Dark Web. Where to buy drugs, guns, stolen credit cards and personal information. Very interesting, somewhat gloomy start to the conference.

Next, the Utah courts, on the home turf, presented their plans for user centered case management. Workgroups in the courts listed what they wanted in improvements from the system. One example: the judicial workspace, a hearing based interface. Judges use it on the bench, it gives them access to the electronic case file and digitally sign orders. Cases can be linked automatically using a person's State Identity Number. MyCase is the interface for the court users. It is designed for the smart phone. An ODR negotiating tool is still in the design stage. Users can subscribe to notifications by email or sms messaging.3rd party interfaces to other government agencies are still in the planning stage. 

I managed to catch the tail end of Digital Design for the User Experience, by Shannon Salter and Margaret Hagan. Shannon is the Chair of the British Columbia Civil Resolutions Tribunal in Canada. Margaret Hagan directs the Legal Design Lab at Stanford Law School. User testing is key: to find testers, you will want to show up where people are already at. Test with the people you are designing for. It is a lot of work to find testers who are not part of the system, but you really need them. Second lesson: it is better to test on a small scale than not at all. Sorry I could not make it to the talk from the start.

David Harvey, Director of the New Zealand Centre for ICT law at the University of Auckland, presented the UK plan for the online courts. It aims to use technology in a disruptive but transformative way, and not mirror existing processes. The universal values of justice need to be observed. I already blogged about this plan in March 2015. Scroll down and you will find it. It will be interesting to see how this ambitious plan is going to materialize.

Singapore Community Justice and Tribunals System (CJTS)

This is an ambitious plan that has already materialized. It was introduced in May 2017. CJTS includes a prefiling assessment to ascertain how the problem can be resolved, and whether it needs to go to courts. Filing including choice of hearing date. The overall design of the procedure is similar to what we in the Netherlands have done for our digital procedures. The claiming party needs to use their Singpass for individuals, Compass for companies, or a CJTS pass for those who have neither. The responding party can choose negotiations, but only up to the hearing date. That gives the negotiation stage some effectiveness. And if negotiations result in an agreement to pay, the parties can apply for a consent order of tribunal.

Singapore is now testing online dispute resolution. They have studied working with artifical intelligence, those papers are on line at the courts web site.

Until now, respondents are not using the system much. The Singapore courts believe this may be because they are not yet familiar with the system.
Good idea I picked up here: the hearing officer, who ensures parties are ready for the hearing before they enter the courtroom. Staff whose work was taken over by the system are retrained for new tasks. 

At the end of a long day, this was a nice surprise: something that is very concrete and actually already works. I look forward to learning more about the effects of this innovative procedure. 

ODR2016

ODR 2016 showed they were taking digitalization seriously, with digital entry tickets and the conference program in a smartphone app. The 15th ODR (Online Dispute Resolution) Conference was held on May 23 and 24th in The Hague in the beautiful conference hall of the Peace Palace. There is a lot of experimenting going on ODR and there are lessons to be learned from those experiments. And because this time the meeting was in the Netherlands, it provided the Netherlands Justice sector with a great opportunity to show what we are doing: what has been realized, what we're working on, and our plans for the future. People were impressed: one participant exclaimed:”I have seen the future, and it is in Dutch!” Here a small collection of my souvenirs from the conference.

Rechtwijzer 2.0

This web site, Rechtwijzer uit elkaar, helps couples wanting to separate make a plan for the divorce and then also arrange for it. Until now, about 250 couples have done so in actual practice. User surveys show that they are satisfied. There is extensive interaction with users to further improve its effectiveness. This tool was developed by HiiL Innovating Justice and Modria dispute resolution systems and the Netherlands Legal Aid Board.  

DemanderJustice: the Uber of the legal world

DemanderJustice (DJ) is a French site where for about €40 people can try to settle a dispute. If this is unsuccessful, it supports bringing the case before a judge for another €70. This works for cases under €10,000, which can be brought to court without a lawyer. At the time of writing, the site had handled more than 250,000 cases. About half were settled; the claiming party won in more than 80% of the cases that were taken to court. The French Bar filed a complaint against DJ for unauthorized exercise of a protected profession. DJ was acquitted of the criminal charge in two instances; the complaints still continue.

Quality and Innovation: Tomorrow’s Judges

KEI  is the acronym for the Dutch courts’ Quality and Innovation program. Working digitally by itself is not enough: it requires a different kind of judge. Tomorrow’s judges will direct cases more actively. The judiciary must become more accessible and more understandable. Supervising judges, in charge of bankruptcy and guardianships, have large amounts of information at their disposal, so they are better able to act on the patterns they discern.

Digital proceedings: learning from experience
Experience with court digitalization, for instance with the eKantonrechter, has produced a lot of questions which are important for the further development of online dispute resolution. By far the most important question is how to involve the defending party in the procedure. Resolving a dispute amicably is often unsuccessful; the claiming party has no choice but to take the case to court. Digital access for the defending party turns out to be a difficult problem everywhere in Europe because digital IDs are not yet good enough to ascertain who enters the court portal and gets access to the case file.

Another question is, how courts can be better geared to the problems they were set up to resolve. 10 years ago, the legal viewpoint was that if someone had a problem, they would take it to a lawyer, and the lawyer would take it to court for you. That viewpoint was wrong. We now know a lot more about what people do they when they have a problem. Solving a problem may well look more like a network or a cloud than a linear chain of events. What does that mean for the administration of justice as a problem solver? And what to do with this observation from a legal aid insurer: our customers want to be more actively involved in the procedure. They are well-informed, the Internet provides them with oil the information they need and they want to be in control of what happens in the case.

I expect there will be more and more people who represent themselves in court; because they are better able to or because they cannot afford legal aid. The Dutch courts will start working on this issue very shortly.

Competition, consecutive or integration?
Participants in the conference could vote about the relationship between ODR and court procedures: Should they compete to improve quality, should ODR be limited to pretrial situations, or should ODR and court procedures be fully integrated? At the end of the conference participants overwhelmingly voted for complete integration. I think this shows we have managed to convince the participants that courts can actually realize digital proceedings.

Experience, rather than plans

The difficulty about things like online dispute resolution is that only users can see how they work. There are logins, and usually a fee to be paid. I had hoped to see a lot of online dispute resolution in actioin gecause I would like to learn to better gear our digital court procedurres to effective ODR. The conference had a series of five-minute pitches, some of which showed their product, but most did not. For the rest, speakers presented a lot of plans. Her Majesty's Online Court (HMOC), part of a large four-year plan in the UK, attracted a lot of attention. I blogged about it last year. Granted, for innovation one needs a plan. However, experience with plans show they almost never realize what they promise. And although they may have some partial results, in the planning phase there are few lessons to be learned from them. 

CTC2015 Day 3

CTC's Day 3 brought the last installment in the track on tools for 21st century judges, but it was actualy about something else. The number of cases in the civil courts is going down, ODR numbers are growing. The US courts have thought about this, and come up with a report on possible solutions. And since technology affects everything, there is a lot of technology in the report.

The panel members were all asked to present their perspective, and prioritize the technology that they felt was most important. For the judge on the panel, it was case management; her insight was that case management is not document management, but actually bringing cases forward. Cases may not all be the same but they are like snowflakes: they are all white, and if there are a lot of them you need to get out of the way, they're just snow.
Push notifications were the favorite of the court manager. The courts CIO's priority was in litigation platforms. His point: courts need to become more user-centric. He referred to the Rechtwijzer 2.0, a site developed by the Dutch Legal Aid Board for couples who want to arrange their own divorce, which he found really cool.

For me, one key question remains: what is the reform going to achieve? Are the courts going to try and retain the segment of cases that is now increasingly handled by private ODR? Will they decide that the courts' core business is resolving disputes that do not have a predictable outcome, as Stephen Breyer, justice in the US Supreme Court once said?

In the endnote, the presenters observed how more and more judges are getting involved. Because my focus was on tools for 21st century judges, I had to miss out on a lot of interesting stuff, like social media and how courts use them and - my other favorite - access to justice. Fortunately, most of the presentations are on line, and sessions were streamed as well, all to be found on ctc2015.org. NCSC, with their limited means, have put together a very interesting, and for the courts, challenging conference. I look forward to CTC2017 in Salt Lake City. 

Day 2: apps, online forms and more judicial tools

Day 2 started with the Midnote. Tara Thomas talked about a lot of apps and their pros and cons. Apps for time saving, for project management, for calendars, conference calls, information sharing, cloud storage of documents. Members of the audience shared their experience with using some of them.

Collin County in Texas shared their own experience in going more paperless. Great insight: electronic document means no more copies, it is always the original. Court staff learned that putting two copies of a document in an electronic case file is unnecessary. 

Williamson County Court judge Bill Gravell explained how they use online forms for self-representing litigants to file their case. The court went from processing 45-50 cases per day to 100, handling time was reduced from 20 minutes to 10 minutes, and instead of 2 jury trials in a day they could now handle jury trials. As cases came to trial faster, defendants chose more often to plead guilty, and the court could drop the jury trial. Bill then introduced me to the people at Tyler Technologies who developed the forms. Tyler actually developed an engine that courts can use to generate their own forms, since they all have different requirements. The engine is a really useful tool. After lunch, I turned my attention to judicial tools again. Three IT staff from Missouri, Wisconsin and Utah shared their experience on developing tools for judges. They had all developed screens with panels presenting different kinds of information: a calendar, pending tasks, quick links to research tools and to Word. Wisconsin gave the judges a private, confidential view of their own statistics: cases pending, cases resolved, that sort of thing. At first, some judges did not want to see it and asked the IT staff to take the panel away. It stayed.

CTC2015 opened on Tuesday morning with a keynote by Mark Britton, CEO of AVVO a platform for legal advice and services. His job was to shake the audience out of any complacency they may have had: the customers are dissatisfied, they are finding other ways to resolve their disputes, your monopoly is not going to help.
The image projected by the movie The Net is that of fear of having personal information on the Internet making you vulnerable to terrorist who want to use it to kill you. 
With Japanese cars and transistor radios for an analogy, he showed how technology disrupts from the consumer up: a new product, for new consumer market: Modria, building an ADR platform, document assembly services like legalzoom; how non-profit legal services are a viable possibility.
Note from the audience: Modria also built a support web site for couples wanting to have an amicable divorce for the Dutch Legal Aid Board.

The Judicial Tools Maturity Model was, for me, the most interesting finding of Day 1. It shows how a court or a judge can grow from paper-based to basic, then to intermediate to an advanced level of IT use. I expect we can use it to test all following presentations on tools for the 21st century judge by their maturity level in the model. From what I saw, the Minnesota courts are moving up in the charts, and the Missouri courts are doing it much more slowly.  

CTC 2015, Minneapolis, Sept 22-24

The 2015 Court Technology Conference is about to start, on Tuesday, September 22. The venue this year is Minneapolis, Minnesota in the United States. This blog is the first of my usual CTC blogs for this year. Keep watching this space for more every conference day. This time, my interest in court technology is even more urgent than in past years. As the product owner of the Netherlands' courts digitalization of the civil procedure, I am acutely aware how difficult developing court IT is. So, I look towards US courts for experience on developing court IT.

Tools for 21st century judges 
The educational track on tools for 21st century judges caught my eye right away. It promises to explore technology tools that can assist judges in making the transition from paper case files to an electronic bench. This transition is very complex as well as critical to maintaining court performance.

Access to justice
Next, I'm always interested in access to justice. As my colleague Martin Gonzales warned repeatedly at CTC2013, more and more people are forced to represent themselves for lack of legal aid. So how can e-filing help self-representing litigants? What other examples do US courts have?

Keynote on Innovation 
The first keynote will address the Innovation Imperative. Mark Britton, founder and CEO of the world's largest community for legal guidance and services, will address innovation. Courts' primary role is to guard the existing legal order. This makes innovation a particular challenge for courts. This keynote opens the conference on Tuesday morning. The keynotes and a lot of other sessions will be streamed live.
Click here for the program: http://www.ctc2015.org 

Flying can be risky

In the last week of August 2015, The European Study Group on Pubic Administration (EGPA) met in Toulouse, France. Eighteen study groups exchange their latest research every year. I presented in the Justice and Court Administration group. Over three days, more than fifteen speakers treated us to a variety of research findings. Amazing, all the things one can research. Some highlights.

Ukraine
Two interesting contributions from Ukraine, both from the United States Agency for International Development (USAID) Fair Justice project. Olga Nikolaieva presented her research on improving alternative dispute resolution in commercial disputes. Sergii Suchenko described how a system for measuring judges ‘workload is being developed, in order to distribute resources better and also to evaluate judges‘ performance.
Sylwia Moravska (Gdansk, Poland) told us how she tries to make court staff treat court users more courteously. She found it hard to answer questions about the goal of her attempt.
   

Knowledge sharing
Knowledge sharing was a hot topic. Jose Ferretti from Brasilia, Brazil gave a presentation that made most of us think: I don’t get it, but I sense he is on to something he cannot yet express in words. That was no knowledge sharing just yet. Sandra Taal from the Netherlands had used her data from other research to test if there are relevant differences in the way men and women share knowledge. There is, but the explanation is rather in women’s more social behavior.

Switzerland
In Switzerland there is some surprising, interesting research.  For instance into the emerging paralegals/legal support staff hire to keep on top of a growing work load. Does it lead to paralegal-justice without access to a judge? Another example: Courts in Switzerland are usually quite small, and they need to deal with different stakeholders, both inside and outside the courts. I had never really thought about this but come to think of it, I recognized some similarities. And then there was the research about regional differences, though to be really large in Switzerland, with all those cantons, languages and cultures. The configurations were different than expected. This research originated because the courts would not participate in it. And now it has been done, they do not want it published.

Finally: Netherlands
in the Netherlands, there is also some interesting research going on. Jos Hoevenaars (Nijmegen) presented his PhD research into the use of the preliminary reference procedure in immigration law. It could produce case law, but it is expensive and it takes a long time. Kars de Graaf and Bert Marseille (Groningen) tested the reasoning behind a proposed law to restructure the system of administrative courts of appeal in the Netherlands. Frans van Dijk, director of the Netherlands Judicial Council, presented a survey by the European Network of Judicial Councils into the perception of judges and citizens of the independence of courts and judges. Independence is perceived most in Denmark, but otherwise, results vary considerably. The image of – for instance – the courts in Poland is very poor.  This survey has produces a lot of data that can keep researchers busy for years to come. The data are open, and so is the report. Finally, I presented the way the courts in the Netherlands attempt to apply lessons from earlier IT-projects in Europe. The key is to manage complexity in all domains by keeping everything as simple as possible. I expect our lessons can be of use to other countries. More knowledge sharing.

Flying can be risky
There were seventeen other study groups. Virtually all those who research public administration in the Netherlands flew back on the same flight. Quite a risk for public administration in the Netherlands.  

As courts go digital: what about online dispute resolution?

Your technology is fifteen years old, a member of the Bar said to me recently, implying we were way behind the times with the technology the Dutch judiciary uses to build its digital procedures. Even if that is right (which it isn’t), that would be a great leap forward from our current systems dating from the 1980s. And yet, he had a point. The new procedure is simpler, but it still looks a lot like the old paper process. A lot of what we do still is a digital version of our paper process. The evolution of the Google car shows we need to develop one step at a time. Moving from a paper based organisation to online dispute resolution, or to an information-based organisation, involves learning to work with new technology. However, we also need to think about what comes after this step. This blog tackles one of the - for traditional courts - most disruptive technologies around: online dispute resolution (ODR).  

As the Dutch judiciary is going digital, the question arises whether this should also include ODR, and whether ODR can replace a face to face hearing in court. Are Skype and FaceTime capable of supporting a debate about the merits of facts and rules, and are people familiar enough with remote communication to grasp new ideas remotely? Or will ODR be suitable for simple disputes only?  

ODR is in the spotlight. In February 2015, the ODR Advisory Group of the Civil Justice Council in the United Kingdom presented Online Dispute Resolution for low value civil claims. Presided by Richard Susskind, the Group examines the field of ODR and makes recommendations for ODR in the legal system of the UK. The report is supported with a wonderful web site  full of background documents, video’s and interviews.  

The system, Her Majesty’s Online Court (HMOC), has two main goals: reduce the cost of dispute resolution and increasing access to justice. Whether or not these two goals are mutually exclusive has been a subject for heated debate for years.
HMOC’s scope will be disputes that are not too complex, and with a value of £ 25.000 at most. It will have three tiers. 

Some first impressions

Tier 1 is mainly about providing information. A web platform can serve to provide this kind of information, unilaterally or through interactive Q&A. My favourite platform is the UK’s Advicenow.org. In the Netherlands, het Juridisch Loket, the government-provided free legal kiosk, is such a platform, and there is interactive Q&A on Rechtwijzer, by the Dutch Legal Aid Board. Modria, the makers of the eBay and PayPal dispute resolution systems, also built Rechtwijzer 2.0. This new Rechtwijzer, now in the final phases of development, is, at present, meant to provide support for couples managing their separation or divorce.
Automated negotiation support in a web platform with user interaction, like Cybersettle, segues into Tier 2. In Tier 2, a human facilitator/mediator can use remote communication, either synchronous such as instant messaging, video conferencing or the telephone, or asynchronous communication like messages or email. Communication with parties is needed to reduce the complexity of the dispute to a level that enables resolving it.   
In Tier 3, the online judge can also use remote communication where needed. This judge also needs a digital case file and secure web communication.
Hazel Genns research taught us that the nature of the problem people have is an important determinant for the type of resolution they require. Consumer problems are resolved easily if the seller has an effective complaints procedure. Problems in long term relationships like family and labour problems are hard to resolve, but also hard to bear, which is an incentive to resolve them consensually. Problems with government agencies tend to be harder to resolve... Our own experience with the eKantonrechter, a simple, consensual court procedure for simple problems, shows that there seems to be little inherent demand for this type of procedure. Consensual dispute resolution has its limits. On the adversary side: low value disputes are not always simple. ODR can be suitable for low value disputes if it can reduce costs, to the user and to the taxpayer, and on the condition that it keeps the complexity of the dispute at an acceptable level – neither over simple nor unnecessarily complex.  

·         Tier One of HMOC should provide Online Evaluation. This facility will help users with a grievance to classify and categorize their problem, to be aware of their rights and obligations, and to understand the options and remedies available to them.

·         Tier Two of HMOC should provide Online Facilitation. To bring a dispute to a speedy, fair conclusion without the involvement of judges, this service will provide online facilitators. Communicating via the Internet, these individuals will review papers and statements and help parties through mediation and negotiation. They will be supported where necessary, by telephone conferencing facilities. Additionally, there will be some automated negotiation, which are systems that help parties resolve their differences without the intervention of human experts.

·         Tier Three of HMOC should provide Online Judges – full-time and part-time members of the Judiciary who will decide suitable cases or parts of cases on an online basis, largely on the basis of papers submitted to them electronically as part of a structured process of online pleading. This process will again be supported, where necessary, by telephone conferencing facilities.

The proposal takes the insights from Hazel Genns Paths to Justice on the needs of people with justiciable problems into account. Genns research was replicated in the Netherlands by Ben van Velthoven and Marijke ter Voert and reported in the dispute resolution delta. This research and its later additions shows that often, information is enough to help people resolve their problem. Sometimes they need someone to help them. If that also fails, there is still the court of law.  

The report does not explicitly analyse the process of dispute resolution, perhaps because there was no judge or magistrate in the Group. Here is what I mean by analysis. Parties first discuss the problem between them. They both introduce information into the debate. Tier 1 can help them with new information suggesting how to resolve their problem. This information can be about ways to resolve problems, but also about legal rules and trends in case law. If they find a solution that satisfies them both, the problem is resolved. If not, they can then choose to use tier 2. In tier 2, the facilitator enters the arena. He or she can help the parties to introduce even more information about their side of the dispute. It is important that the information the parties used in Tier 1 is also still at hand. Next, all that information can be reduced to what is relevant for resolving the problem. Some negotiation or mediation may be useful to help parties find a solution. If that fails as well, the online judge in Tier 3 can decide the dispute with a judgment.

This brief description provides some indication for the way in which information technology can be leveraged to resolve disputes.

Back to my initial questions

Hazel Genns research taught us that the nature of the problem people have is an important determinant for the type of resolution they require. Consumer problems are resolved easily if the seller has an effective complaints procedure. Problems in long term relationships like family and labour problems are hard to resolve, but also hard to bear, which is an incentive to resolve them consensually. Problems with government agencies tend to be harder to resolve... Our own experience with the eKantonrechter, a simple, consensual court procedure for simple problems, shows that there seems to be little inherent demand for this type of procedure. Consensual dispute resolution has its limits. On the adversary side: low value disputes are not always simple. ODR can be suitable for low value disputes if it can reduce costs, to the user and to the taxpayer, and on the condition that it keeps the complexity of the dispute at an acceptable level – neither over simple nor unnecessarily complex. There is more to ODR than just an alternative to a face to face court hearing. When a dispute needs a fresh look at the merits of facts and rules, and possibly a new idea of the stakes involved, a face to face court hearing may still be the most effective way of resolving it. As remote communication becomes more mainstream and cheaper, remote hearings may also become more mainstream.