Concept for AI
metamorworks / stock.adobe.com
2023-02-13 publication

Interview with ChatGPT

When is an artificial intelligence (AI) good, bad or even dangerous? And how does an AI itself assess this issue? An interview with the chatbot ChatGPT.

Contact
VDE dialog - the technology magazine
Downloads + Links

VDE dialog: Is it possible to measure the quality of artificial intelligence? 

Chatbot ChatGPT: There are several ways to measure the quality of artificial intelligence (AI). One way is to use metrics that measure the performance of an AI system with respect to a specific task or data set. Another way is to use benchmarks that measure the performance of an AI system relative to other systems. There are also other metrics and benchmarks that are specific to the particular applications of AI, such as the Perceptron criteria that measure the quality of classification of images or the quality of machine translation. It is important to note that measuring the quality of AI should not be limited to a single metric or benchmark, but rather a combination of different metrics and benchmarks.

Can an AI be qualitatively good, but still be bad?

An AI can be good on quantitative metrics, but still be bad – if it does not meet expectations or requirements on qualitative aspects. For example, an AI system may have high accuracy on a particular task, but still exhibit undesirable behaviors in the process, such as discriminatory behavior toward certain groups. Or an AI system may perform well in certain environments or situations, but perform poorly in other environments or situations.

How can this be prevented?

It is important to consider not only quantitative metrics, but also qualitative aspects when assessing the quality of an AI system. This can include reviewing how the system behaves in relation to ethical and societal concerns, reviewing how robust and secure the system is, and reviewing how well the system performs in unforeseen circumstances.

Muss noch mehr getan werden, um die Qualität künstlicher Intelligenz zu verbessern? Does more need to be done to improve the quality of artificial intelligence?

There is still much that can be done to improve the quality of AI. Some of the key areas where further efforts should be made are:

  1. Ethics and responsibility: It is important that ethical and responsible AI systems are developed. This includes avoiding discriminatory behavior and considering societal and ethical implications of AI.
  2. Robustness and security: AI systems must be robust and secure to ensure that they function correctly in unforeseen circumstances and are protected from attacks.
  3. Data and annotations: AI systems are only as good as the data on which they are trained. It is important to collect and annotate high-quality and representative data to ensure that AI systems work for a wide range of applications.
  4. Transparency and explainability: It is important that AI systems are transparent and explainable so that the decisions they make can be understood and to build trust in the system.
  5. Interdisciplinary collaboration: the development of AI requires the collaboration of experts from different fields such as engineers, scientists, ethicists, social scientists, lawyers and others.

Overall, there are still many challenges and opportunities in AI research and development, and more efforts are needed from scientists, industry, regulators, and society to improve the quality of artificial intelligence.

What are the dangers if an AI is of inferior quality?

Then there are some dangers that can be significant for both the use of AI in companies and for society as a whole. Some examples are:

  1. Faulty decisions: An AI system that is of inferior quality can make faulty decisions that can have a negative impact on individuals, companies, or even society.
  2. Discriminatory behavior: Such an AI system may exhibit discriminatory behavior, which can lead to disadvantages for certain groups.
  3. Security risks: Such an AI system can be vulnerable to attacks and thus endanger the data and the IT infrastructure.
  4. Loss of trust: Such an AI system may affect user and public trust in the technology, which may affect the acceptance and use of AI.
  5. Financial losses: Such an AI system can also cause financial losses for companies, e.g., due to faulty decisions that can lead to production losses or losses on the market.

Therefore, it is important that AI systems are of high quality to ensure that they meet expectations and requirements and minimize risks. And that is why it is also important to carefully monitor and improve the quality of AI systems.

Konzeptionelle Technologie anhand einer Illustration von kuenstlicher Intelligenz
kras99 / stock.adobe.com

How do you rate your own quality?

As an artificial intelligence, I am trained to perform specific tasks and answer questions. I am a large language model trained on a variety of texts, and I am able to perform human-like language processes.

I try to answer all questions as accurately and informatively as possible, based on the knowledge available to me. However, I am also a computerized service and may make mistakes, especially with complex or specific questions. I ask for your understanding if my answers are inaccurate or incomplete.

Can you pose a danger too?

As an artificial intelligence, there is a risk that I – like any other computer system  can be manipulated or exploited by attackers to perform malicious activities. However, it is important to emphasize that I am a purely passive system. I have no autonomy and no agenda of my own. I merely respond to the requests that are made to me. I will only pose a threat if someone intentionally or unintentionally puts me in such a role.

However, it is also important to consider the potential impact of AI on society and ethics in general, especially when it comes to the application of AI in sensitive areas such as health, finance and security. AI systems need to be carefully developed and monitored to ensure that they are ethical and responsible and that they do not have an undesirable impact on society.

Last question: Have you heard about the new "AI Quality & Testing Hub" to be built in Hesse?

I have no information that such a hub exists or is planned in Hesse. However, it is possible that there is a new project there that I am not aware of and that deals with this topic. This is because my knowledge is limited to a specific point in time and cannot refer to current developments in Hesse.

Editor's note: The conversation was held at the end of January. Whether the chatbot ChatGPT has heard anything about the AI Quality & Testing Hub in the meantime can be checked at https://chat.openai.com – at least when the chatbot has free capacity again. Recently, it has been overrun with requests.

Artificial intelligence in the quality check

Bildgenerator DALLE-E 2

Bei der Bebilderung dieses Artikels half der Bildgenerator DALLE-E 2  von Open AI. 

| https://openai.com/dall-e-2/
2023-02-13 publication

Germany has a great interest in measuring and thus proving the quality of artificial intelligence. Only then can the "Made in Germany" seal also work for products developed here in this field. A new "Ai Quality & Testing Hub" in Hesse wants to pioneer this - and establish itself as a strong partner for companies and developers.

By Martin Schmitz-Kuhl

Read more