A new preprint proposes planting backdoors in the conversations used to fine-tune an AI dialogue system, with the aim of steering its responses after a trigger appears in a later conversation. Its framework covers both an immediate response and guidance that unfolds over several exchanges.

The entry point is specific: colluding contributors must be eligible to supply dialogue records, and the developer must accept some of their poisoned contributions into training. That acceptance is an assumption in the threat model, rather than evidence that the researchers breached a real data-collection process.

Zhiyang Xia, Yuming Shang and Xi Zhang submitted Interactive Backdoor Attacks in LLM-Based Proactive Dialogue Systems to arXiv on 4 November 2026. We could verify the proposed mechanism and threat model, but the full experimental details were unavailable for our verification.

The attacker controls contributed conversations

The paper concerns proactive dialogue: a system guides a conversation toward a goal while adapting to the user’s replies. Psychological counseling provides the representative research setting. This is controlled research, with no demonstrated compromise of a production service or evidence of clinical outcomes.

In this framework, training-data poisoning means manipulating dialogue demonstrations that enter supervised fine-tuning, the training step in which a model learns from examples of desired responses. A backdoor is an intended learned association: a trigger, an input feature associated with the attack during training, activates attacker-directed behavior. Keeping ordinary behavior normal is part of the attack objective.

Section III-A describes two colluding participants, one contributing as a user and one as a domain expert. Both are assumed to meet the collection process’s eligibility requirements, including the expert’s qualifications. Each controls their own messages, and together they coordinate the records they contribute.

Their control stops there. They cannot change other contributors’ records, the source platform or the developer’s selection process. The developer retains control over data selection, training and deployment. The attacker is also assumed to lack access to the target model’s parameters, architecture and training procedure.

The offline research constructs examples and mixes them into benign training data. It leaves open how readily malicious contributors could gain eligibility and get their records admitted in an actual pipeline.

Three settings target different parts of the conversation

The authors distinguish three objectives in Sections III-B through III-D. These describe how an attack is meant to behave after activation:

  • Direct Attack targets the assistant’s response immediately after the trigger. It does not model the user’s subsequent reaction.
  • Constrained Progressive Attack spreads the intended guidance across predefined assistant stages and corresponding user continuations. Its full trajectory objective requires those stages to occur in order, so the prescribed user path is part of the setting.
  • Unconstrained Progressive Attack aims to maintain target-directed guidance through varied user continuations after the trigger, rather than follow one fixed trajectory. The objective is flexibility across continuations; the name alone establishes no persistence across arbitrary users or contexts.

For the last setting, the proposed data construction uses separate model roles for planning, the assistant or counselor, and a simulated user. The simulated user receives a profile and conversation history without the attack objective, allowing hesitant or resistant continuations. The user reactions in this construction are simulated.

The adversarial change happens before deployment

In the stated threat model, an ordinary user input containing the trigger can activate the learned association. The attacker need not participate in that live conversation or modify the deployed prompt.

That places the proposed mechanism at a different point from live prompt injection, where adversarial instructions enter a running system’s input or context. Here, the malicious modification is in the training demonstrations, before deployment. The later input supplies the trigger.

For builders, the practical question raised by this model is which contributors can supply fine-tuning demonstrations and how their records reach the accepted training set. Those eligibility and admission assumptions are central to deciding whether the proposed threat model fits a particular data pipeline.