Jasper Leo

Jasper Leo

Gain and loss are equal.

Research 🔬

[RL] DAC: The Double Actor-Critic Architecture for Learning Options

November 22, 2022

Summary, ISLab, BUPT, Beijing

The option framework is reformulated as two parallel augmented MDPs. under this new formulation, all policy optimization algorithms are readily available for learning intra-option policy, termination policy, and master option. we apply AC algorithms on each augmented MDP and The DAC architecture is designed. Combined with the PPO algorithm, an empirical study is conducted on challenging robot simulation tasks.

[ML] A Path Towards Autonomous Machine Intelligence

November 22, 2022

Summary, ISLab, BUPT, Beijing

Different researchers have different perspectives on the ideas proposed by LeCun, but in general this paper is extremely enlightening and leading. The paper proposes an architecture and training paradigm for building autonomous intelligences, combining concepts such as configurable predictive world models, behavior driven by intrinsic motivation, and hierarchical joint embedding architectures trained by self-supervised learning.

[LM][AGI] WebGPT: Browser-assisted question-answering with human feedback

November 13, 2022

Summary, ISLab, BUPT, Beijing

The study fine-tuned the GPT-3 large model to achieve long-form QA by manipulating browser search, i.e., giving long and meaningful complete answers to open-ended questions. The result is that more than half of the generated results are more satisfying than the answers given by humans, with higher accuracy and information validity.

[LM] The Debate Over Understanding in AI’s Large Language Models

November 10, 2022

Summary, ISLab, BUPT, Beijing

The article describes what LLM is, how it is trained and works, and how it works, and then shows that LLM is also brittle and can make mistakes when disturbed. The article points out that there are two opposing voices, one arguing that this is the budding of general intelligence, and the other arguing that LLM only learns the form of language rather than its meaning. The article also shows, with some examples, that shortcut learning, a phenomenon often invoked in machine learning, is also present in LLM, i.e., learning systems that rely on spurious correlations in the data.

[LM] Challenging BIG-Bench tasks

November 10, 2022

Summary, ISLab, BUPT, Beijing

CoT was already available when BIG-Bench first came out, but CoT did not perform well on small scale models (emergent effect could not be achieved), so BIG-Bench did not mention using CoT; but after that, PaLM / Davinci-002 / code-davinvi-002 and other larger scale models appeared. So there was a motivation to verify the effect of CoT on the new baseline of BIG-Bench. Sure enough, CoT is indeed better for many tasks.

[RL] Option-Critic

November 04, 2022

Summary & Implementation, ISLab, BUPT, Beijing

We have read an important paper on option-based hierarchical reinforcement learning, The Option-Critic Architecture, and have critically reviewed the derivation of the equations to understand the SMDP process in option-state augmented space and the corresponding algorithmic framework.

[LM] Chain-of-Thought

October 23, 2022

Summary, ISLab, BUPT, Beijing

CoT is a form of discrete cue learning. More specifically, contextual learning of language models adds many textual logical representations of thought in between compared to the previous traditional contextual learning (i.e., a series of texts as input for the large model to complete the output).

[LM] Mind’s Eye

October 23, 2022

Summary, ISLab, BUPT, Beijing

The research proposes a new paradigm for large models to implement reasoning about the physical world Mind’s Eye, by getting information from real simulations of physical problems in the Mujoco physics engine, and inputting auxiliary information along with the problems themselves to enable large models, the improvement in UTOPIA baseline is significant and can be done so that small models plus simulations can outperform large models after the effect improvement is significant.

[LM] Out of One, Many

October 15, 2022

Summary, ISLab, BUPT, Beijing

Mainly prompt experiment, no large deal. Working on bias and skewed margin, now a popular field.

[KG] A Survey on Knowledge Graphs

October 15, 2022

Summary, ISLab, BUPT, Beijing

An overview of knowledge graph principles, KGE, knowledge acquisition, and major applications is presented.