Apple Machine Learning Research Foundation models are trained on large-scale web-crawled datasets, which often contain noise, biases, and irrelevant information. This motivates the use of data selection techniques, which can be divided into model-free variants — relying on heuristic rules and downstream datasets — and model-based, e.g., using influence functions. The former can be expensive […]Continue reading

Apple Machine Learning Research Multimodal Large Language Models (MLLMs) have demonstrated a wide range of capabilities across many domains, including Embodied AI. In this work, we study how to best ground a MLLM into different embodiments and their associated action spaces, with the goal of leveraging the multimodal world knowledge of the MLLM. We first […]Continue reading

Apple Machine Learning Research Modern wearable devices can conveniently record various biosignals in the many different environments of daily living, enabling a rich view of individual health. However, not all biosignals are the same: high-fidelity biosignals, such as photoplethysmogram (PPG), contain more physiological information, but require optical sensors with a high power footprint. Alternatively, a […]Continue reading

Apple Machine Learning Research This work was done in collaboration with Swiss Federal Institute of Technology Lausanne (EPFL). Image tokenization has enabled major advances in autoregressive image generation by providing compressed, discrete representations that are more efficient to process than raw pixels. While traditional approaches use 2D grid tokenization, recent methods like TiTok have shown […]Continue reading

Apple Machine Learning Research We examine the capability of Multimodal Large Language Models (MLLMs) to tackle diverse domains that extend beyond the traditional language and vision tasks these models are typically trained on. Specifically, our focus lies in areas such as Embodied AI, Games, UI Control, and Planning. To this end, we introduce a process […]Continue reading

MIT News – Artificial intelligence All biological function is dependent on how different proteins interact with each other. Protein-protein interactions facilitate everything from transcribing DNA and controlling cell division to higher-level functions in complex organisms.Much remains unclear, however, about how these functions are orchestrated on the molecular level, and how proteins interact with each other […]Continue reading

MIT News – Artificial intelligence While early language models could only process text, contemporary large language models now perform highly diverse tasks on different types of data. For instance, LLMs can understand many languages, generate computer code, solve math problems, or answer questions about images and audio.   MIT researchers probed the inner workings of LLMs to […]Continue reading

MIT News – Artificial intelligence Biology is never simple. As researchers make strides in reading and editing genes to treat disease, for instance, a growing body of evidence suggests that the proteins and metabolites surrounding those genes can’t be ignored.The MIT spinout ReviveMed has created a platform for measuring metabolites — products of metabolism like […]Continue reading

MIT News – Artificial intelligence Proteins are the workhorses that keep our cells running, and there are many thousands of types of proteins in our cells, each performing a specialized function. Researchers have long known that the structure of a protein determines what it can do. More recently, researchers are coming to appreciate that a […]Continue reading

MIT News – Artificial intelligence The MIT Stephen A. Schwarzman College of Computing has received substantial support for its striking new headquarters on Vassar Street in Cambridge, Massachusetts. A major gift from Sebastian Man ’79, SM ’80 will be recognized with the naming of a key space in the building, enriching the academic and research […]Continue reading