Investigators: Daniel Andr´es D´ıaz-Pach´on (PI) and J. Sunil Rao (co-PI).
Summary
When David Wolpert and William MacReady published their No Free Lunch Theorems (NFLT) that on average no search does better than a blind search, there was disquiet in the computer science community because these results imply that there is no one-size-fit-all algorithm that can do well in all searches, throwing away the dream of a “theory of everything” in machine learning. Wolpert and MacReady concluded that, in order to solve each particular problem, it was necessary to incorporate “problem-specific knowledge into the behavior of the algorithm”. More specifically, the NFLT say that no search works better on average than a blind search, i.e., a search according to a uniform distribution.
We therefore propose the following two aims:
- Aim 1. To develop a new test based on (2). Notice that even if an inference to programmer activity is not considered, at the very least the rejection of the null hypothesis H0, that the target T happened according to a probability given by a maxent distribution, entails rejecting that the endogenous distribution of the space was the maxent distribution. Therefore an explanation is needed; in fact, Edwin Jaynes made a similar comment, highlighting that such deviations from maxent could also lead to new discoveries. Thus actinfo works also as a call to accountability.
- Aim 2. To implement actinfo as a measure of performance between different learning strategies. There is an antecedent in unsupervised learning to the striking situation illustrated by Wolpert: recently we used actinfo in order to show that, contrary to usual wisdom, the principal components of smallest variance, instead of those with higher variance, lead to optimal estimates of mode regions in multivariate distributions that are symmetric and unimodal. In this situation, the target was the region of infimum volume with a given probability β. In the same way, by following the formalization proposed by Wolpert for supervised learning, we plan to implement actinfo to determine the performance between different supervised learning strategies.