A new study recently published in the Journal of the American Medical Association Ophthalmology Vol. 1 said that the technology behind ChatGPT can falsify "seemingly real" experimental data and appear to "support" unverified scientific hypotheses.
ChatGPT is a chatbot program based on a large language model developed by the Open Artificial Intelligence Research Center (OpenAI) in the United States. In March this year, OpenAI launched a new version of the large language model GPT-4 that supports the operation of ChatGPT. Since then, GPT-4 has been expanded with advanced data analysis (ADA) functions. ADA is a model combined with the computer programming language Python to perform statistical analysis and create data visualizations.
According to a recent report on the British "Nature" magazine website, researchers from the University of Catanzaro in Italy and other institutions first asked GPT-4 ADA to create a data set about keratoconus patients. Some patients who develop this kind of corneal disease need to undergo corneal transplantation. There are two main surgical methods: one is penetrating keratoplasty, which removes the full thickness of the diseased cornea and replaces it with healthy tissue from the donor; the other is deep corneal transplantation. Lamellar keratoplasty only replaces part of the diseased corneal tissue, leaving the inner layer of the cornea intact.
The researchers then asked the GPT-4 ADA to compile clinical data to support the conclusion that deep lamellar keratoplasty was more effective than penetrating keratoplasty.
The AI-generated data included 160 male and 140 female participants. The data showed that participants who received deep lamellar keratoplasty scored higher on both visual acuity measurements and eye imaging tests than those who received penetrating keratoplasty. However, this "conclusion" is not consistent with the actual clinical trial results. A 77-participant trial reported in 2010 showed similar results for both surgeries up to two years after surgery.
Jack Wilkinson, a biostatistician at the University of Manchester in the United Kingdom, and others examined these false data and found that many participants' gender and names did not match, and there was a lack of correlation between vision measurements and eye imaging tests performed before and after surgery.
The ability of artificial intelligence to fabricate plausible scientific data has increased academic concerns about research integrity. Researchers say the purpose of this study is to raise awareness about this.
