We have a Steam curator now. You should be following it. https://store.steampowered.com/curator/44994899-RPGHQ/

[BG3] Changing NPC Voices using AI

Game development hub. Projects, modding, and resources.
Post Reply
User avatar
orinEsque
Posts: 1614
Joined: Oct 9, '23
Location: Narnia
Gender: Potato

[BG3] Changing NPC Voices using AI

Post by orinEsque »

Work in progress

Table of contents
  1. Prerequisites
  2. Extract Voices
  3. Train your own Voice Model
  4. Changing voices using AI

[back to index]
1. Prerequisites
This guide assumes you know how to use BG3 Multimodders tool (BG3MT). If not, read Getting Started with BG3 Modding.

1. Download Bg3 AI Voice Toolkit
  1. Link to Repository
    How to download a github repo:
    Image
  2. Extract the contents of the zip file. This will be a folder named: bg3-ai-voicetoolkit-main
2. Download Mangio-RVC-v23.7.0
  1. Get the INFER_TRAIN version of Mangio-RVC-Fork - Download
  2. Extract the contents into bg3-ai-voicetoolkit-main folder.
  3. Delete all the audio files inside: bg3-ai-voicetoolkit-main\Mangio-RVC-v23.7.0\audios
3. Norbyte's Lslib Tool
  1. Download Norbyte's Lslib Tool - Download
  2. Extract the contents into the bg3-ai-voicetoolkit-main/lslib folder
4. BG3 Multimodders Tool
  1. Open BG3 Multimodders Tool (BG3MT)
  2. Go to Utilities > Game File Operations > Unpack Game Files
    Image
  3. Check the Voice.pak option and click Confirm.
  4. In directory that you extracted BG3 Multimodders Tool to, there should now be a folder called UnpackedData
  5. Inside there will be a "Voice" folder.
  6. Cut/Copy this "Voice" folder and paste it inside bg3-ai-voicetoolkit-main
  7. Your bg3-ai-voicetoolkit-main folder should now look like this:
    Image
5. Audacity
Download and install Audacity from https://www.audacityteam.org/


[back to index]
2. Extract Voices by voice ID

Find voice ID of your NPC
  1. Let's say I want to extract Lorroakan's voice. I recall he said the lines. "You've been to Shar's temple"
  2. Download the Asset dictionary
  3. Look for "You've been to Shar's temple". Copy it's corresponding "handle". In this case it would be "h069a339fgff56g40bag9ccdg55a1599a0ade"
  4. Open BG3MT and search for this handle.
  5. In the results there should be a Soundbanks/*.lsf file
  6. Extract and Open the file in notepad++
  7. Search for Lorroakan's dialogue handle within the lsx file, you should see: UUID_yourhandle.wem
  8. Everything before the underscore is your voice id, in this case "va9d4b71db0ef429e82106dc8be986ee9"
Extract an NPC's voice lines
  1. Open bg3-ai-voicetoolkit-main\lib\voice_selector.py in notepad++
  2. You'll see a numbered list with character names and an id
  3. Simply maintain the same format to add another character.
  4. "Name" can be of your choosing that best describes the NPC. In this case: Lorroakan
  5. id should match the voiceID of the NPC you are interested in.
  6. Now add your NPC's voiceID as an "id" to the voice_selector.py. In Lorroakan's case it's:
    Image.
    Note: Make sure you number this correctly. There was 37 voice ID's before this one. So this one is number 38.
  7. Go back and open 1.Select Character.bat and you will see that Lorroakan's voice lines are now selectable. And audio files have been extracted to bg3-ai-voicetoolkit-main\Mangio-RVC-v23.7.0\audios


[back to index]
3. How to Train your own Voice Model

Prepping the voice
Let's train Lorroakan's voice.
Good quality audio will always create better quality models. To keep things simple let's train ingame dialogue audio because they are, by nature, very high quality and doesn't have background noise.
  1. Lorroakan's voice was extracted to bg3-ai-voicetoolkit-main\Mangio-RVC-v23.7.0\audios
  2. Open Audacity
  3. Grab around a 100 of Lorroakan's voice lines and drop it in Audacity
  4. Press ctrl + A to select all the audios in Audacity (mine is v3.4.2) .
  5. Go to Effect > Special > Truncate Silence
  6. Configure the settings to look as follows:
    Image
  7. Click Apply
  8. Go to File > Export Audio
    Image
  9. Set the export Folder to be the full path to your bg3-ai-voicetoolkit-main\Mangio-RVC-v23.7.0\datasets\Lorroakan folder
    Note: you should change "Lorroakan" to your NPC or voice actor name
  10. Once you've got the setting to look like the image above. Click Export.

Training the Model
  1. Open bg3-ai-voicetoolkit-main\Mangio-RVC-v23.7.0\go-web.bat
  2. You should see a command prompt. Let it do it's thing, and soon a page will open in your browser.
  3. Go to the "Train" tab.
  4. Now follow the tutorial below




[back to index]
Change Voices using AI
A few things have changed since the video below was made.


1. Correct Conversion
Hidden Content
This board requires you to be registered and logged-in to view hidden content.

2. Packing your Voice Mod
There is a slight difference in packing voice mods versus regular mods. In BG3MT, you need to set Compression to "None"
Image

3. Video Tutorial of remaining steps
If you followed the tutorial so far and installed everything the prerequisites & extracted your NPC's voice ID, you can skip any installation steps in the video below. I have timestamped where you need to start:


For learning purposes, you can choose touse preexisting a voice models. There's a huge list in this link:

Voice Models: https://huggingface.co/QuickWick/Music- ... /tree/main

The pth files go in weights folder. and the .index file should go in a custom folder with the same name as the model within logs.
e.g.
weights/britney.pth
logs/britney/britney.index
Last edited by orinEsque on April 10th, 2024, 00:41, edited 25 times in total.
OnTilt
Top
User avatar
Xenich
Posts: 1171
Joined: Feb 24, '24

Post by Xenich »

@orinEsque

Will this work with other projects as well?

Basically, I have unpacked all the The Chronicles Of Myrtana Archolos ".Wav" files, which are currently in another language.

My thoughts were using AI to convert them to English.

I haven't played with this much and it seems like the tools here "may" help in this, but since you have experience, I figured you might be able to point me in the right direction.

There are two approaches I have considered.

1. Use AI to translate the audio and use whatever generic voices they have for that process. it appears programs like Maestra, ElevenLabs, etc... may be able to handle this to some extent.

2. Unpack all text translation script and run it through AI using a similar approach with text to speech and then selecting various voices (maybe training as is shown in this guide).

What would you suggest is the best approach or are there any tools you have used which you know of that might be better suited for this objective?

Edit:

Ok, been messing with this a bit. Maestra works, is simple enough (just upload the wave file, pick your voice and "style", but sounds very "generated". ElvenLabs is better, allowing for more adjustments for a natural sound, but won't accept watermarked wav files unfortunately. I really dislike these online tools as it isn't practical for any large projects. There are over 8k wav files which, unless they allow for bulk upload and conversion, this is going to be a painstaking procedure. I will keep looking for an offline software, but not sure if there are any currently (I would imagine in time as AI becomes more commonplace, you will be able to find software with pack files to download).

Anyway, if I could do a bulk conversion, this would be a very quick change. It would allow for a complete language change to the game without too much effort. Just convert them, repack them and done.

Even if the voices are a bit corny and generic, It would still be acceptable I think, Gothic was never known for its voice overs being great.

Anyway, if you have any recommendations, I would appreciate it.
Last edited by Xenich on May 13th, 2024, 16:07, edited 1 time in total.
Top
Post Reply

Return to “The Foundry”