Stupid Computer

People seem very keen on pointing out in a discussion about AGI that:

1. There is something like intelligence
2. Given enough computational time with a given function there should be a result that is complete. The Church-Turning thesis.
3. The fact that intelligence is something material there must be a function that describes that intelligence.

There is just a couple of problems with this conviction:

1. The efficiency problem. Church-Turing does not imply the calculation to be efficient on any time scale. If I throw some amino acids into a sufficiently warm puddle I might be able to create intelligence. I just have to wait a couple of million years. At least for this experiment we know it had one successful outcome.
2. Let us accept that the world is deterministic. In the sense that if we know the position, direction and speed of every particle in the universe at a given time t we would be able to predict the position of every particle at time t+1, t+2 .. t+n and thus would be able to completely predict the universe. I am not trying to drag Heisenberg’s uncertainty theorem in here. It is not relevant. If we know the position of every particle at t-1 and the position of every particle at t we could do the same. For those on the religious or philosophical disposition. This neither excludes God nor a free will. Our real problem is that we cannot store that information anywhere or do calculations with it without requiring to keep track of that storage and calculation itself or keep it outside the physical universe we are trying to predict. That is logically impossible. What we do is make models: Simplified versions of reality that help us predict some specific future event. Those models are incomplete and thus wrong, but sometimes useful. Now we strive to make better models which means better predictions, but with every particle we do not take into account our uncertainty about the outcome is growing. I hope one does also see that given a model being a simplified version of reality throwing more data at it won’t do much unless it is the data relevant to our simplification. A calculation is just such a model, so it will necessarily be incomplete. It might still be useful, but given that this calculation does not exist yet, we will have to see.
3. A computational function describing intelligence might have a logical problem in itself. Math is a product of our intelligence. Can it ever be used to describe that intelligence itself? It seems less obvious than it appears at first glance. Moreover, if we could compress enough of our intelligence into useful symbols to do math with do we need an AGI? I am on the fence on this one, hence the question marks.
4. Even if we managed to create a mathematical function describing intelligence it does not mean we have AGI. My carefully calculation of he trajectory of a ball might be completely correct, but no ball was moved in the process. The other side of the same argument is that we can make calculations that are completely valid as calculations, but far larger than our physical world would allow: The c in E=mc^2 is not easily reached if you are not a photon. A googol is a completely valid number in math, but nothing in the physical world can be counted as such. The mathematical function might simply not fit in our physical universe.
5. Nearly forgot, we might not be able to determine such a mathematical function computable because of the Halting probl.

As an afterthought and because some people keep suggesting they are related: LLM’s are not an attempt to find that calculation that describes intelligence in the sense the Church Turing thesis implies. It is a small calculation of a single relation between tokens that we combine between millions of token in het hope that this simulates intelligence. There is no mathematical justification or proof for making those combinations other than probability. Don’t get confused about the fact that probability is expressed in math. High probability is itself not a mathematical proof. The Riemann hypothesis is probably true, but that high probability is not proof in itself.