Machine learning and privacy
Moderator: Úlfar Erlingsson, Research Scientist, Apple
13:00-14:30 UTC
Privacy stands out as the quintessential success story in the quest to design robust machine learning. So far it has not been possible to design machine learning algorithms that always generalize or always perform well despite distribution shifts, but we do have machine learning algorithms that provide provable bounds on memorization of private information from the training set, thanks to differential privacy. Come to this session to discuss this and other privacy-related topics, such as multi-party computation, homomorphic encryption, model theft, and real-world applications of privacy preserving technology.
|
Imitation learning
Co-moderators:
Tapani Raiko, Principal Research Scientist, Apple
Tobias Gindele, Machine Learning Engineer, Apple
17:00-18:30 UTC
Imitation learning resembles both generative modeling and reinforcement learning. As with generative modeling, the goal is to learn a model that can reproduce complicated patterns (in this case, patterns of behavior) from training data. As with reinforcement learning, the goal is to learn an agent that can successfully carry out prolonged interactions with its environment. Imitation learning is an especially active area of research today, with both open loop approaches that learn purely from offline data, and closed loop approaches such as GAIL and SQIL that actively involve the environment in their learning process.
|
Beyond Modeling: Challenges to Practical Applications of ML to Healthcare
Moderator: Olivia Koshy, ML Operations Engineer, Nines
18:00-20:00 UTC
Without a doubt, the field of machine learning has made tremendous progress across a wide variety of tasks and fields. We have seen this especially in the context of integrating advances from academic research into industry settings. Yet rarely have we seen the same level of success when specifically applied to the healthcare field. Why is that?
This session aims to deep dive into the obstacles of ML healthcare applications in a real world setting. We'll reflect on the failure cases of the past and obstacles for the future with the following starting points:
-
Our inputs - how can we assemble datasets which are useful for training clinically viable models? And how does this compare to the status quo (academic datasets e.g.)?
-
The model development cycle - how can we decide what models would provide clinical value and then build them?
-
Our outputs - how do we deploy and productionize, integrating our results with the clinical workflow and evaluating impact on standards of care?
|
Doing 'cognitive neuroscience' on models - will it help us understand generalization?
Moderator: Catherine Olsson, Senior Program Associate, Open Philanthropy Project
20:30-22:00 UTC
We would like to trust that machine learning systems will generalize safely to new environments. Unfortunately, training environments are “underspecified”: many different underlying strategies for solving a task can have equally good training performance, and only reveal themselves to be problematic when deployed in the real world.
If we had better tools for understanding how a neural net "thinks", could we audit models' "reasoning" in advance of deployment?
Different methods in the literature could be described as doing "cognitive science" on models, treating the neural nets themselves as the object of study. These include "neuroscience"-like approaches that open the black box of the neural net (including probing methods in NLP; and distill.pub-style work studying individual neurons' activations) and "cognitive"-like approaches that carefully craft novel inputs to test hypotheses (such as the texture-biased ImageNet images used by Geirhos et al, or the modified robust and non-robust datasets used in "Adversarial Examples Are Not Bugs, They Are Features")
Are these "cognitive neuroscience" approaches potentially useful for auditing models in advance of deployment to avoid unintended generalization? Or is this a dead end?
|