How do open data and AI fit together?

Artificial intelligence has long been a topic of discussion among Germany’s local authorities. One promising approach: combine AI with open data in order to offer users added value. At the 13^th Open Data Network Meeting on January 18, 2024, the speakers shed light on how this can be achieved. 

Almost 100 participants were on hand as Christian Stocker from Switzerland kicked off the event by discussing how open data and AI have been linked in Zurich.  As a developer at the web agency Liip, Stocker worked with his colleagues to create ZüriGPT (zuericitygpt.ch). The site can answer many questions related to the city, such as: What is Zurich doing to combat climate change? How do I get a building permit?

The starting point for the project, which the agency carried out on its own without the participation of local government, was the less-than-optimal search engine on the city’s website. “When you have 50,000 HTML pages, finding what you’re looking for can be a bit tedious,” Stocker explained.

Stocker also developed LinkedDataGPT (ld.gpt.liip.ch), a tool that uses AI to simplify queries of the linked data made available by the city of Zurich. Based on OpenAI’s GPT-API, questions in natural speech are converted into SPARQL queries. This allows less technically adept people to obtain the information they are looking for from the city’s open data.

The tool has access to the relevant datasets and files, which it draws on to create the answers. “People put their knowledge at the tool’s disposal and ultimately I determine which question is answered and what the answer looks like based on the knowledge provided,” Stocker said, describing the process. The required information was obtained by crawling the website beforehand. In some cases, the data had to be adapted. For example, ChatGPT was unable to find information on waste collection, since it was stored in a PDF, so Stocker and his team converted it into a compatible format.

Stocker and his colleagues then developed the app for various cities including Lausanne, as well as the for parliament’s website in the canton of Valais. But there are limits to what the tool can do. As Stocker openly admitted, a demo version for the Zurich Zoo had a few bugs. For example, the answer to the question “When is the lion active?” contained two contradictory statements. Answers should be checked regularly as a result, he said.

The second speaker, Justus Engelland, reported on how the city of Brunswick combines artificial intelligence and open data. Engelland is deputy director of the city’s Digitalization Department and the person responsible for the topic of open data (opendata.braunschweig.de).

The city’s open data portal has been up and running since 2022 and currently contains 58 datasets. “AI is becoming increasingly important in public administration,” Engelland said. “This is because administrators have a lot of texts and data at their disposal, so the focus is shifting to AI-based data analysis.”

Engelland described three specific use cases: geodata on aircraft noise; budget data; and sensor data on foot traffic in Brunswick’s city center. The city’s administrators used the software DataAnalyzer to evaluate the underlying information. In the first case, the data was used to create a map depicting aircraft noise, which was then transferred to a regional map including the airport’s surroundings. DataAnalyzer pooled the necessary data.

The second example looked at the city’s budget data. The underlying data was retrieved from the open data portal as a CSV file. DataAnalyzer then extracted the budget data. Pie charts were subsequently used to visualize individual expenditures. The application makes it possible to ask follow-up questions, such as how the city can potentially save money.

DataAnalyzer was able to evaluate daily foot traffic in Brunswick and display it as a diagram. “Thanks to AI and dialogue-driven data evaluation and presentation, it’s possible to analyze things faster,” Engelland said.

According to Engelland, one aspect which offers enormous potential is that unrelated files can be merged even if they are formatted differently. Unlocking the full potential that AI-based data analysis has to offer will require standardized data processing, he said. The quality of the input data and queries (known as prompts) determines the quality of the output. Engelland also stressed that in some cases it is still necessary for humans to check the results.

Finally, Stefan Riedel from Technologie Campus Grafenau presented a guide that helps individuals and organizations in the tourism industry make data openly available (tourismus.bayern).

In a second brief talk, Luis Moßburger from byte, the Bavarian Agency for Digital Affairs (open.bydata.de), presented a handout for local administrators on the topic “High-value datasets.”