Dialog style transfer with parameter-efficient fine-tuning
DOI:
https://doi.org/10.17721/1812-5409.2025/2.30Keywords:
natural language processing, parameter-efficient fine-tuning, low-rank adaptation, text generationAbstract
This work explores the application of modern fine-tuning methods, characterized by high parameter efficiency, to the task of generating dialogue responses in the style of a specific fictional or real-life character. The main goal of the study is to develop an approach that allows existing language models to be adapted to the specific communication or speech style of a particular personality using a minimal number of new parameters. This avoids the need for full retraining of large models and reduces computational costs.
One of the key objectives is to enable the implementation of a compact yet functional language model that can be run on a standalone computer without requiring servers or powerful graphics processors. Such an approach opens new possibilities for personalized or entertainment applications, such as creating chatbots capable of interacting with users in the persona of a character from a book, movie, or even a historical figure.
To implement this approach, we propose the use of the LoRA (Low-Rank Adaptation) method, which allows efficient fine-tuning of existing transformer architectures — in particular, the GPT-2 model — without retraining all of its layers. Using LoRA makes it possible to add stylistic features to the model’s speech without compromising its core ability to generate logical and coherent texts.
To evaluate how well the model has learned to imitate the target speech style, we additionally train a BERT model as a classifier to distinguish between texts written in the desired style and ordinary, neutral responses. This provides an objective quality metric for generation and allows us to assess the effectiveness of the fine-tuning.
The dataset used for training was generated using the GPT-4 mini model, which enables the rapid creation of a large number of examples in the desired style.
Pages of the article in the issue: 193 - 196
Language of the article: Ukrainian
References
Hu, Edward J., Shen, Yelong, Wallis, Phillip, Allen-Zhu, Zeyuan, Yuanzhi, Li, Wang, Shean, Lu, Wang, & Chen, Weizhu. (2021). Lora: Low-rank adaptation of large language models. arXiv. https://doi.org/10.48550/arXiv.2106.09685
Hypersniper. Philosophy dialog [Dataset]. Hugging Face. https://huggingface.co/datasets/Hypersniper/philosophy_dialogue
Keskar, Nitish Shirish, McCann, Bryan, Varshney, Lav R., Caiming, Xiong, & Socher, Richard. (2019). Ctrl: A conditional transformer language model for controllable generation. arXiv. https://doi.org/10.48550/arXiv.1909.05858
Pfeiffer, Jonas, Kamath, Aishwarya, Rücklé, Andreas, Kyunghyun, Cho, & Gurevych, Iryna. (2020). Adapterfusion: Non-destructive task composition for transfer learning. arXiv. https://doi.org/10.48550/arXiv.2005.00247
Radford, Alec, Wu, Jeffrey, Child, Rewon, Luan, David, Amodei, Dario, & Sutskever, Ilya. (2019). Language models are unsupervised multitask learners. OpenAI blog. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
Sanh, V. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv. https://doi.org/10.48550/arXiv.1910.01108
Schick, Timo, & Schütze, Hinrich. (2020). It's not just size that matters: Small language models are also few-shot learners. arXiv. https://doi.org/10.48550/arXiv.2009.07118
Zhang, Tianyi, Kishore, Varsha, Wu, Felix, Weinberger, Kilian Q., Artzi Yoav. (2019). Bertscore: Evaluating text generation with bert. arXiv. https://doi.org/10.48550/arXiv.1904.09675
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Ruslan Pravosud, Oleksandr Marchenko
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
