The implementation of AI and automation in the public sector is aimed at improving efficiency and freeing up staff time, both in public-facing systems and behind-the-scenes tools. However, caution is necessary to avoid falling for overhyped claims as missteps can have serious consequences for citizens and public bodies alike.
A wide variety of public sector functions are currently being automated, showcasing the diverse applications of these technologies. Examples include using chatbots in local authorities, automating social care assessments, and employing data analytics to identify at-risk families. It is important to note that these systems do not impact everyone equally and can have significant consequences for marginalized communities, as well as potential biases and privacy violations.
There are risks associated with public service automation that must be carefully considered. Automated systems must be rigorously tested for their ability to achieve desired outcomes, as failures can lead to inefficacy and adverse impacts. Biases in automated systems often stem from the data used to train them, resulting in unfair outcomes in cases like predictive policing tools and the 2020 A Level algorithm.
Concerns about data sources, citizen consent, transparency, and accountability also surround automation in the public sector. Establishing guiding principles for automation, such as interrogating the need, assessing impact versus risk, setting ‘red lines,’ ensuring transparency, upskilling staff, and involving citizens in decision-making, can help balance the benefits and risks of automation.
Trust from citizens is crucial in the successful implementation of automated systems in the public sector, as seen in examples like the NHS’s digital transformation efforts and the Post Office/Horizon scandal. Without a thoughtful and principled approach to automation, citizens’ rights and trust may be compromised. Careful consideration is necessary to leverage the benefits of automation without sacrificing citizen rights or trust.