Gaps in the Foundations of Planning with Approximation

Planning consists of imagining courses of action and their consequences, and deciding ahead of time which ones to do. In the standard reinforcement-learning model of an intelligent agent, the component that does the imagining of consequences is called the transition model of the environment, and the deciding in advance is via a change in the agent’s policy. Planning and model learning have been studied for seven decades and yet remain largely unsolved in the face of genuine approximation—models that do not become exact in the high-data limit. In this talk I briefly assess the challenges of extending dynamic-programming-style planning (value iteration) in the five most important ways: function approximation, partial observability, stochastic transitions, average reward, and temporal abstraction (options). My assessment is that these extensions are straightforward until they are combined with genuine approximation in the model, in which case we have barely a clue how to proceed in a scalable way. Nevertheless, we do have a few clues; I suggest the ideas of expectation models, metadata, and representation search as general strategies for learning approximate environment models suitable for use in planning.

Recording

Combining task with motion planning

Multi-goal task planning has been an optimization question in Operation Research and Logistics for a long time. We will present novel advances for high-speed solving these discrete planning problems. With fast single-source shortest path search, the map is condensed into a graph of customer orders.

While the discrete multi-goal task planning problem is already hard, the "physical" motion planning problem is even more demanding. The integration of task and motion planning is considered one the most important problems in robotics nowadays. Robots have sizes, heading, and velocity, and their motion can often be described only according to non-linear differential equations. The dynamics of movements, existing obstacles and many waypoints to visit are only some of the challenges to face. In real-world problems, we often have additional constraints like inspecting areas of interest in some certain order, while still minimizing the time for the travel. The trickiest part is to solve the hard combinatorial discrete tasks like the generalized and clustered traveling salesman problems, and - at the same time - providing valid trajectories for the robot. We extend a framework in which a motion tree is steadily grown, and abstractions to discrete planning problems are used as a heuristic guidance for the on-going solution process to eventually visit all waypoints. In case of inspection, we generate the waypoints fully automatically, using a combination of skeletonization methods together with a filtering mechanism based on hitting sets.

Robots used for inspection or moving of goods are also often required to visit certain locations subject to time and resource consumption. This requires not only planning collision-free and dynamically feasible motions, but also reasoning about constraints. To effectively solve this open problem, we couple task planning over a discrete abstraction with sampling-based motion planning over the continuous state space of feasible motions. The discrete abstraction is obtained by imposing a roadmap that captures the connectivity of the free space. We increase the expressiveness and scalability of the approach, as we raise the number of goals and the difficulty of satisfying the time and resource constraints. With our technology we are able to efficiently generate and execute long-term missions in real-time. The robot, the environment model, and the planning problem specification can be modified non-intrusively, essential in many application scenarios. Other topics of interest are robot planning with limits in energy consumption.

Recording

From AI in Robots to AI in Finance: Planning Examples and Discussion

After many years of research in academia on AI and autonomous robots, for the last three years, I have been the head of AI research at JPMorgan. During all this time, I have looked at many challenges with an AI approach, addressing knowledge, representation of states, actions, behaviors, planning, multiagent interactions, learning. In this talk, I will share several interesting problems that we have encountered and solutions that we devised in AI in Robots and AI in Finance. I will focus on planning examples and will discuss novel ideas in planning, execution, and learning, which have the potential of great applicability and great impact in real domains.

Recording

AI for Housing

Marketplace KE Holdings Inc. (“Beike”) is a leading integrated online and offline platform for housing transactions and services. The company is a pioneer in building the industry infrastructure and standards in China to reinvent how service providers and housing customers efficiently navigate and consummate housing transactions, ranging from existing and new home sales, home rentals, to home renovation, real estate financial solutions, and other services. In this talk, we will discuss the challenges and opptunites for AI in housing marketplace, especially modern housing platforms like Beike. We will introduce how AI technologies have been applied to analyze unique data (including data for property, housing customers and agents) on Beike’s platform, to improve service efficiency and customer experience.

Recording