“Multilayer neural networks are among the most powerful models in machine learning, yet the fundamental reasons for this success defy mathematical understanding.”
Source: Proceedings of the National Academy of Sciences 2018
'Artificial Intelligence' (AI) systems predominantly use Neural Networks to achieve their 'Machine Learning' capability. However :
“No underlying principle has guided the design of these learning systems, other than vague inspiration drawn from the architecture of the brain (and no one really understands how that operates either).”
Source: Quanta Magazine Sept. 2017
A theory known as the 'Information Bottleneck Method' which was first proposed in 1999, is currently being evaluated as a possible explanation - but as yet there is no general agreement on how the networks operate.
A further complication is that the 'back propagation' technique which artificial neural nets use doesn't have a direct equivalent in biological neural nets. Biological neural nets have however been shown to locally back-propagate their electrical action potentials. The function(s) of this phenomenon is also currently unexplained.
“While there is ample evidence to prove the existence of backpropagating action potentials, the function of such action potentials and the extent to which they invade the most distal dendrites remains highly controversial.”
Editor's note: This is a quite rare example of a modern man-made technological system which clearly works, but that no-one as yet understands. (Although there are many examples of man-made medicines which work in as-yet-unexplained ways - see the medicine section for examples.)