It’s always funny seeing arguments like this as someone with a computer science education. A lot of people act like you can’t have anything complex unless some intelligent being deterministically writes a lot of if-else statements to implement it, which requires them to know and understand in detail what they are implementing at every step.
But what people don’t realize is that this is not how it works at all, there are many problems that are just impractical to actually “know” how to solve yet we solve them all the time, such as voice recognition. Nobody in human history has ever written a bunch of if-else statements to be able to accurately translate someone’s voice to text, because it’s too complicated of a problem, no one on earth knows how it works.
Yet, of course, your phone can do voice recognition just fine. That is because you can put together a generic class of algorithms which find solutions to problems on their own, without you even understanding how to solve problem. These algorithms are known as metaheuristics. Metaheuristics fundamentally cannot be deterministic, they require random noise to work properly, because something that is deterministic will always greedily go in the direction of a more correct solution, and will never explore more incorrect solutions, whereby an even better solution may be beyond the horizon of many incorrect ones. They also do have to be somewhat deterministic as well, because you need some greed or else the random exploration would be aimless.
A simple example of a metaheuristic is that of annealing. If you want to strengthen a sword, you can heat up the metal really hot and let it slowly cool. While it’s really hot, the atoms in the sword will randomly explore different configurations, and as it cools, they will explore less and less, and the overall process leads them to finding rather optimal configurations that strengthen the crystaline structure of the metal.
This simple process can actually be applied generally to solve pretty much any problem. For example, if you are trying to figure out the optimal route to deliver packages, you can simulate this annealing process but rather than atoms searching for an optimal crystaline structure, you have different orders of stops on a graph searching for the shortest path. The “temperature” would be a variable that represents how much random exploration you are willing to accept, i.e. if you alter the configuration and it’s worse, how much worse does it have to be for you to not accept it. A higher temperature would accept worse solutions, at very low temperatures you would only accept solutions that improve upon the route.
I once implemented this algorithm to solve sudoku puzzles and it was very quick at doing so, and the funny thing is, I’ve never even played sudoku before! I do not know how to efficiently solve a sudoku puzzle, I’ve honestly never even solved one by hand, but with sudoku it is very easy to verify whether or not a solution is correct even if you have no idea how to find the solution and even if finding it is very difficult, verifying it is trivially easy. So all I had to do is right the annealing algorithm so that the greedy aspect is based on verifying how many rows/columns are correct, and the exploration part is just randomly moving numbers around.
There are tons of metaheuristic algorithms, and much of them we learn from nature, like annealing, however, there’s also genetic algorithms. The random exploration is done through random mutations through each generation, but the deterministic and greedy aspect of it is the fact that only the most optimal generations are chosen to produce the next generation. This is also a generic algorithm that can be applied to solve any problem. You can see a person here who uses a genetic algorithm to teach a computer how to fly a plane in a simulation.
Modern AI is based on neural networks, which the greedy aspect of them is something called backpropagation, although this on its own is not a metaheuristic, but modern AI tech arguably qualifies because it does not actually work until you introduce random exploration like a method known as drop out whereby you randomly remove neurons during training to encourage the neural network to not overfit. Backpropagation+dropout forms a kind of metaheuristic with both a greedy and exploratory aspect to it, and can be used to solve just about any generic problem. (Technically, ANNs are just function-approximators, so if you want to think of it as a metaheuristic, the full metaheuristic would have to include all the steps of creating, training, and then applying the ANN in practice, as a metaheuristic is a list of steps to solve any generic problem, whereas an ANN on its own is just a function-approximator.)
Indeed, that’s how we get phones to recognize speech and convert it to text. Nobody sat down and wrote a bunch of if-else statements to translate speech into text. Rather, we took a generic nature-inspired algorithm that can produce solutions for any problem, and just applied it to speech recognition, and kept increasing the amount of compute until it could solve the problem on its own. Once it solves it, the solution it spits out is kind of a black box. You can put in speech as an input, and it gives you text as an output, but nobody really even knows fully what is going on in between.
People often act like somehow computers could not solve problems unless humans could also solve them, but computers already have solved millions of problems which not only has no human ever solved but no human can even possibly understand the solution the computer spits out. All we know from studying nature is that there are clever ways to combine random exploration and deterministic greed to form processes which can solve any arbitrary problem given enough time and resources, so we just implement those processes into computers and then keep throwing more time and resources at it until it spits out an answer.
We already understand how nature can produce complex things without anyone “knowing” how it works, because we do that all the time already! You do not need a sentient being to tell the beetle how to evolve to fit into its environment. There is random exploration caused by genetic mutations, but also a deterministic greedy aspect caused by “survival of the fittest.” This causes living organisms to gradually develop over many generations to something fit for its environment. And life has had plenty of time and resources to become more suited to its environment, life has been evolving for billions of years, with the whole resources of the planet earth and the sun.
It’s always funny seeing arguments like this as someone with a computer science education. A lot of people act like you can’t have anything complex unless some intelligent being deterministically writes a lot of if-else statements to implement it, which requires them to know and understand in detail what they are implementing at every step.
But what people don’t realize is that this is not how it works at all, there are many problems that are just impractical to actually “know” how to solve yet we solve them all the time, such as voice recognition. Nobody in human history has ever written a bunch of if-else statements to be able to accurately translate someone’s voice to text, because it’s too complicated of a problem, no one on earth knows how it works.
Yet, of course, your phone can do voice recognition just fine. That is because you can put together a generic class of algorithms which find solutions to problems on their own, without you even understanding how to solve problem. These algorithms are known as metaheuristics. Metaheuristics fundamentally cannot be deterministic, they require random noise to work properly, because something that is deterministic will always greedily go in the direction of a more correct solution, and will never explore more incorrect solutions, whereby an even better solution may be beyond the horizon of many incorrect ones. They also do have to be somewhat deterministic as well, because you need some greed or else the random exploration would be aimless.
A simple example of a metaheuristic is that of annealing. If you want to strengthen a sword, you can heat up the metal really hot and let it slowly cool. While it’s really hot, the atoms in the sword will randomly explore different configurations, and as it cools, they will explore less and less, and the overall process leads them to finding rather optimal configurations that strengthen the crystaline structure of the metal.
This simple process can actually be applied generally to solve pretty much any problem. For example, if you are trying to figure out the optimal route to deliver packages, you can simulate this annealing process but rather than atoms searching for an optimal crystaline structure, you have different orders of stops on a graph searching for the shortest path. The “temperature” would be a variable that represents how much random exploration you are willing to accept, i.e. if you alter the configuration and it’s worse, how much worse does it have to be for you to not accept it. A higher temperature would accept worse solutions, at very low temperatures you would only accept solutions that improve upon the route.
I once implemented this algorithm to solve sudoku puzzles and it was very quick at doing so, and the funny thing is, I’ve never even played sudoku before! I do not know how to efficiently solve a sudoku puzzle, I’ve honestly never even solved one by hand, but with sudoku it is very easy to verify whether or not a solution is correct even if you have no idea how to find the solution and even if finding it is very difficult, verifying it is trivially easy. So all I had to do is right the annealing algorithm so that the greedy aspect is based on verifying how many rows/columns are correct, and the exploration part is just randomly moving numbers around.
There are tons of metaheuristic algorithms, and much of them we learn from nature, like annealing, however, there’s also genetic algorithms. The random exploration is done through random mutations through each generation, but the deterministic and greedy aspect of it is the fact that only the most optimal generations are chosen to produce the next generation. This is also a generic algorithm that can be applied to solve any problem. You can see a person here who uses a genetic algorithm to teach a computer how to fly a plane in a simulation.
Modern AI is based on neural networks, which the greedy aspect of them is something called backpropagation, although this on its own is not a metaheuristic, but modern AI tech arguably qualifies because it does not actually work until you introduce random exploration like a method known as drop out whereby you randomly remove neurons during training to encourage the neural network to not overfit. Backpropagation+dropout forms a kind of metaheuristic with both a greedy and exploratory aspect to it, and can be used to solve just about any generic problem. (Technically, ANNs are just function-approximators, so if you want to think of it as a metaheuristic, the full metaheuristic would have to include all the steps of creating, training, and then applying the ANN in practice, as a metaheuristic is a list of steps to solve any generic problem, whereas an ANN on its own is just a function-approximator.)
Indeed, that’s how we get phones to recognize speech and convert it to text. Nobody sat down and wrote a bunch of if-else statements to translate speech into text. Rather, we took a generic nature-inspired algorithm that can produce solutions for any problem, and just applied it to speech recognition, and kept increasing the amount of compute until it could solve the problem on its own. Once it solves it, the solution it spits out is kind of a black box. You can put in speech as an input, and it gives you text as an output, but nobody really even knows fully what is going on in between.
People often act like somehow computers could not solve problems unless humans could also solve them, but computers already have solved millions of problems which not only has no human ever solved but no human can even possibly understand the solution the computer spits out. All we know from studying nature is that there are clever ways to combine random exploration and deterministic greed to form processes which can solve any arbitrary problem given enough time and resources, so we just implement those processes into computers and then keep throwing more time and resources at it until it spits out an answer.
We already understand how nature can produce complex things without anyone “knowing” how it works, because we do that all the time already! You do not need a sentient being to tell the beetle how to evolve to fit into its environment. There is random exploration caused by genetic mutations, but also a deterministic greedy aspect caused by “survival of the fittest.” This causes living organisms to gradually develop over many generations to something fit for its environment. And life has had plenty of time and resources to become more suited to its environment, life has been evolving for billions of years, with the whole resources of the planet earth and the sun.
This is the kind of answer I can appreciate.
Thanks.