Language Immersion Virtual Environment (LIVE)
Create a virtual environment that leverages the GPT API for Language Learning!
Objectives
Your character has moved to another part of the world and must now interact with the locals in a foreign language to solve simple missions. Mail a letter, shop for groceries, go to the doctor, find the bus station… The player is given a mission and must figure out how to interact with the non-English speaking NPC's to complete it. NPC's will give directions, instructions, answer your questions, describe things, sell you things, etc.
Main Objectives:
- Everything is entirely immersive! No English spoken (or typed) here!
- You will likely want to buy models and entire scenes on the Unity or Unreal asset store.
- Something similar to this: https://assetstore.unity.com/packages/3d/environments/urban/polygon-city-low-poly-3d-art-by-synty-95214
- Can get reimbursed up to $100 per person for assets or GPT API access.
- Create a simple town scene populated with a number of Sims-like NPC's that you can interact with. This is probably best done with a top-down perspective. 2D or 3D work equally well, but 2D is simpler.
- The player will be given simple missions: Buy bread, find the bus station, walk the dog in the park, take a taxi to someplace…
- The system should be able to figure out when you have completed a mission and give a score boost or something.
- You type whatever you want to say. Probably need to enable the keyboard language options.
- The NPCs will say realistic things. If they don't understand you they will ask questions.
- All NPC output is GPT-powered. NPC's should act and speak like their characters.
- This may be useful (for Unity): https://www.youtube.com/watch?v=FxkL5htl-to
- To keep things simple for this first version, the NPC's don't do anything. Just stand around and wait for you to talk to them.
- Note that the GPT API can be very slow at times. Prepare to have to work around this (test mode does not use GPT, test your work with canned output, etc).
Stretch Goals:
- Enable text-to-speech for the NPC's and/or for you.
- Make the environment as realistic as possible. Cars drive around. NPC's go about their daily lives instead of just standing there.
- The system could be beefed up to understand language learning progression. Start with very simple missions and progress from there.
- Have a hint system of some kind.
- Add badges (micro-credentials) or other incentive system like game currency.
Motivations
Language learning the only way that truly works!
Qualifications
Minimum Qualifications:
A desire to create the greatest Language Learning tool of all time!
Preferred Qualifications:None Listed
Details
Project Partner:
Samarendra Hedaoo
NDA/IPA:No Agreement Required
Number Groups:1
Project Status:Accepting Applicants
Keywords:WebMobileUnityAPIUnreal EngineGPT