If you try in ChatGPT:
“what is the numeric value of 2 * ( 5 * 2 ) replacing ‘*’ by the addition operator?”
You will mostly get wrong answers.
A simulation of 25 questions / answers using davinci 3, gets 50% of the answers right and wrong.
Answer: ’20’ – Count: 13
Answer: ‘2 + 5 + 2 = 9′ – Count: 9
Answer: ’22’ – Count: 2
Answer: ’14’ – Count: 1
Results show how the machine always remembers the expression but in half of the answers it has forgotten the basic premise.