Bot Colony
AMA on Bot Colony on r/games
https://www.reddit.com/r/Games/comments/5y80in/were_north_side_creators_of_bot_colony_currently/
Bot Colony Investigation (Major Update)
THE FIRST REAL CONVERSATION GAME
While our new upgrade took (much) longer than anticipated, we hope it was worth the wait. Bot Colony now delivers on the promise to be the first real conversation game. We’re added 2 - 3 hours of unique content in the form of a new investigation mission. Bot Colony was originally built on the gameplay of commanding robots to manipulate objects. But we wanted to achieve more, and we’ve been hard at work on our language pipeline. The game has now evolved to the point where you can also investigate in a casual conversation style. This level of communication between the player and the game was much harder to achieve and required much better language understanding than just commands. Virtual assistants like Siri, Google Assistant or Amazon Echo aren’t yet able to carry the kind of multi-step conversations that you’ll have with Jimmy as you play the new mission. This said, we’re not claiming that Jimmy will understand everything you say – there are still many challenges ahead.
THE INVESTIGATION MISSION
You will need to discover how a secret prototype chip got into the house of a Nakagawa scientist and why the house was empty when you arrived. You investigate using your own words - not dialogue trees! If you ask good questions about what Jimmy observed, you’ll be rewarded with videos of his observations. There are 14 such videos for you to discover, and you get to see a few during training – so don’t skip training! You’ll practice investigating the events that took place over 48 hours, using questions like ‘what happened then’, ‘...next’, ‘...before that’, ‘...after that’, and editing the questions you asked previously and resubmitting them, so you don’t need to type too much.
We recommend that you play through typing. First, you’ll need to ask LOTS of questions, so you run the risk of getting hoarse if you play through voice. To support the investigation mission, our software needs to understand what you ask about people, places, times and dates. This is difficult enough as is – without having to also deal with the errors introduced by speech-to-text.
NEW FEATURES
As you play, you’ll see tangible proof of our commitment to take the Bot Colony conversational experience to the next level:
- you can refer to what was said previously by scrolling the conversation window (F10).
- the game will keep you informed of your progress discovering the story (green means you’re done exploring a time period, yellow means you need to keep going until it turns green).
- many questions which Jimmy used to answer with "I don't know" now get answers, so you’ll be able to learn about the family members, their lives and relationships, and various events.
- When an exact answer is not available, you’ll still get an interesting hint relevant to your question
- With this upgrade, every robot in Bot Colony gets its own voice and accent – which adds to the texture of the experience.
- A noticeable frame rate improvement
- The game website www.botcolony.com was entirely redesigned. We added a new section on the background story and world design – a kind of a graphic story of how Bot Colony evolved. We’ve also published of all of Henry Fong’s concept work (Henry now works for Disney). Finally, the October 2007 original storyboard is online (the game is indeed ambitious – 2017 will mark the 10 year anniversary from the birth of the game!).
GOING FORWARD
In closing, a few words about our plans for Bot Colony going forward:
- Following requests, we've committed to support Achievements in the next upgrade.
- Robots in Arrival don’t understand what you say as well as Jimmy does in Intruder (and he could still do better :). We plan to improve this - it's the core experience of our game.
- We’ve made progress on episode 3, Riot. We’ll post updates on www.botcolony.com/blog.
BUY button is back!
We have assured Valve of our commitment to keep Bot Colony going (we did keep the servers going after all :), and the BUY button is back as of today. Your beloved Jimmy - and his cousin Kate - are now busy moonlighting - take a look at www.northsideinc.com - it's kind of neat that the technology powering Bot Colony found its way into other applications, I'm hoping we'll do better with that than we did with Bot Colony. Personally, I still hope to see this game finished, all 12 episodes of it!
We've made a big upgrade in January, the Robot Visual Memories. We plan to make additional upgrades to conversation with robots in the future, but I cannot commit to specific dates. Early Access terms apply - there are no promises of evolution.
We'd be very happy to work with developers interested to take on continued development of Bot Colony on a revenue sharing basis - this makes more sense now as the game seems to have more traction after it 'died' than it had before. If you know any sharp developers, pass the word around.
We have the assets ready for the next 3 - 4 episodes, as you can see from the 30 min video that tells the Bot Colony story using game assets (look for The Story Untold video on the Store page). If our new venture works out, I hope we'll find the money to finance continued development of Bot Colony - that's our heritage and that's how we got into Natural Language Understanding - and it would feel great to finish what we started.
Thank you again!
Community Support Keeps Bot Colony Alive!
What would help us the most is you spreading the word about Bot Colony.
We’ve started working on in-house servers as a long-term solution. When that is in place we’ll talk to Steam about putting back the BUY button on Bot Colony.
Since the update, we’ve fixed missing background facts in Intruder and Charlie being stuck in Arrival. We've added checkpoints to Arrival so you won’t lose your level progression. Robot Visual Memories (RVM) offers (by far) the most interesting gameplay in Bot Colony - make sure you play it! You need to find out what happened since Jimmy arrived, and why the house is now empty. The tech behind RVM is described here http://gamasutra.com/blogs/EugeneJoseph/20140626/219765/Natural_Language_Understanding_and_TexttoAnimation_in_Bot_Colony.php
Currently, few players get to RVM. Why? You get the RVM mission AFTER your score for Intruder is displayed and Arrival is unlocked - so most move on to Arrival. CHEAT: you can actually start playing RVM right after dealing with the police Bot; if the owner comes home and catches you, just re-start from the last checkpoint (every 3 objects).
The Bot Colony experience is defined by how good Natural Language Understanding (NLU) is, so improving NLU is #1 for us. Lots of NLU work is still needed to make the robots in Arrival smarter. Mike's navigation has many problems, but you should be able to get him to do what you need. With enough sales, we may even be able to deliver more content (Riot and beyond – watch the Story Untold video from our Store page).
Here are TIPS FOR PLAYING that everyone buying eBooks or Steam keys receives. Following these tips will result in a better game experience.
TIPS FOR PLAYING
1. What makes Bot Colony unique is speaking (or typing) with robots who understand what you say, as long as it's related to their task or memories. You'll command robots to overcome game challenges. You'll also need to explore their memories to piece together what happened.
a) When you train speech-to-text, you read material that contains useful information on how to talk to robots. Bottom line: use complete English sentences to be understood in Bot Colony.
b) Don't skip the tutorial in Intruder.
c) Ask "What should I do now?" if you're not sure what to do at some point. In Arrival, talk to Miki.
d) (INTRUDER) Ask Jimmy "What do you know?" and then "What commands do you know?" to be reminded what commands a robot knows. Following popular demand, here are Jimmy's commands:
- Go to place (Go to the living room, go to the vase). He will go, and turn to face an object, if mentioned.
- Pick up object (Pick up the vase). Does face, reach and grab, below. You can ask "What do you hold?".
- Drop object
- Face object (not exposed, part of pick up X)
- Reach for object (part of pick up X)
- Grab object (part of pick up X)
- Push in object (push in the cushion). Close the drawer (or the cushion :) works.
- Put object1 on object2 (put the red box on the blue box). Put object1 to the left/right of object2.
- Put object1 between objects (put the vase between the candles, put the bottle between the sinks). Put object1 in the center of object2.
- Rotate object by Z degrees clockwise/counterclockwise
- Swap object1 with object2. Put object1 where object2 was - also works.
- Align object1 with object2 (for pot, television, chair - imagine you're on a plane or ship looking FORWARD; you'll have a red light on your left and a green one on your right. The object you align with is the plane :)
- Open door (open cupboard door - in the kitchen)
- Close door (or guitar case)
- Point to object (or point to room)
- Move forward/back (by Y meters)
- Turn clockwise/counterclockwise ( by Y degrees)
- Stop to reset a robot.
- wave, jump, nod
- What do yo see? - it's useful to look though his camera to see inside the chest in Hideki's room)
Cindy can follow me or stop following me and Mike moves up/down or goes to a shelf. You can combine the commands above to form new ones - this can be great fun! Try teaching 'mess up the room' and post a screenshot of the result. Steam keys await upvoted screenshots :)
e) Referring to things: Differentiate 1) using ORDER (to "Which vase? The one on..., or the one on...?" reply "the first one" or "the second one"), 2) using COLOR ( Which game box? 'the blue one', 'the purple one)', 3) using the OBJECT under (pot in kitchen, "the one on the stovetop"), 4) using OPEN/CLOSED state (for the drawer, cushion 'the open one').
f) Robot Visual Memories are 14 video segments recorded by Jimmy that tell a story. The basic investigative questions to get to these videos are below. Once you get an answer, it sets the time-context for your next questions, so you don’t have to always specify the time and date (unless you want absolute precision). Here are some useful questions to ask Jimmy:
Who is X?
What do you know about X?
How do you know that?
When did you first/last see X?
What did X do at HH:MM on Day/Date? (example: What did Ayame do at 20:15 on Thursday?) What did Ayame do then? What did Masaya do next? What did X do before that?
What happened then? What happened before/after that? What happened at HH:MM on (day of week)? (What happened at 11:30 on 26/08/2021?) – this will work even if after/before don’t return more facts because Jimmy doesn’t look back/forward far enough.
What happened to X? What happened to X at (time) on (date)/(day of week)?
When did X arrive/enter/leave the house? When did you arrive? Where did X go after that?
What did X say at (time) on (day)? What did X say before/after that?
Where was X at (time) on (day of the week)/date?
Other questions that will help you find out more about the characters, like :
What does X like? When did X play? When did Y speak on the phone?
•Leave Dialogue history (F10) open during RVM gameplay. Use the middle mouse button to scroll through the dialogue text, use Up Arrow to repeat a previous question and hit Enter.
g) Use Help! Use UP arrow to access previous commands. Use F10 to see dialogue history ( you can COPY PASTE from it - so you can share problems and funny things robots say with the community!. Use F1 to talk to Miki in Arrival. F3 gets you out of mediated mode (talking to a robot through a tablet or console).
h) (Arrival episode) Consider teaching Mike a new command made up of basic commands (for example, 'scan briefcase') to save precious time and stay alive.
2. It's best to play initially by typing - this will prevent Speech-to-text errors and will give our NLU a better chance to work well.
Once you know what you can expect from NLU, please feel free to experiment with Speech-to-text.
3. If you enjoy the game, you can help keep it alive by reviewing it on the Bot Colony Steam page under Helpful Customer reviews (I hope you'll Recommend it!).
Please tell your friends about Bot Colony, very few people heard about our game.
*** Steam keys: A review on the Store page gets you one - email the author to claim it. *** Posting funny dialogue that gets upvoted (use F10, COPY, PASTE), same*** *** Posting unexpected dialogue behaviour, same *** Screenshots of messed up rooms, same ***
What's Next and The Untold Story video
A gift to our fans, Bot Colony - The Untold Story was uploaded to Steam recently (it's the 3rd video from the left on our Store page). This 30 minute video was made to give a potential developer a complete view of the game, to help estimate the effort required to complete it. The video goes through all 12 episodes, using prototype assets and stills. A bit clinical (given the intended audience), the video does give a good idea of the scope of Bot Colony, and how much we had advanced before we had to stop. I hope that people who watch it will agree that Bot Colony can be a lot more than just talking to boring drones.
The game now has quite a bit to offer. In the new update, you piece together a juicy story from video clips a domestic robot recorded (RVM’s). You discover these video clips through conversation – truly unique gameplay not offered by any other game. In the two episodes released, you command robots through language under time-constraints – also unique gameplay. With the Untold Story video and the eBook – the Bot Colony story now has closure and can be enjoyed 'as is', like a released game. If Bot Colony continues to attract attention - it's now rated as popular content on Steam, who knows?
What next?
We were encouraged by your response and the emergence of a community of fans. We'll look at the possibility to keep the game going by hosting it in-house on our own servers. Unfortunately, we had to lay off 40 of our employees, so we're now considering the possibility of building a server farm from their PC's. We'll also need to move to a much smaller place, so we'll have to stack them high :)
Thank you again for supporting Bot Colony - something really new in gaming!
Pulling the plug on Bot Colony
To give you some figures, the NLU technology powering Bot Colony cost around $20M and many years to develop. It is probably one of the best NLU technologies available in the world today. The game itself probably cost another $2M - $3M. On the other hand, the lifetime sales of the game to date are about $10K - not even one day of operating costs for us when we were fully staffed (we worked on it for 7 years). The update that we launched last Friday cost about $460,000 to develop. Incidentally, this update features some really innovative, unique gameplay, and I urge those who bought the game to try it - we worked really hard on it. When we released the update on Friday, we made public a video showing what we had achieved in Episode 3. Unfortunately, all of this only brought in a measly $200 in gross sales (we went up from 793 lifetime units to 814). This was the last straw that led to the decision today to shut the game down - 96 hours after launching the update. I've subsidized Bot Colony for years, and as a result I now owe $5M to the banks. My house - and it's a nice house that I love where I've lived the last 15 years - is on the market. I've had to lay off 40 of our employees - most of our staff - because of the lack of sales. It is a tragedy, especially because we've put our hearts into this game, believing we will change the industry by enabling players to converse with the characters in games.
I've always enjoyed taking care of our customers, so I cringe at the thought of pulling the plug, but we don't have a choice. A customer wrote:
This is an awesome update. Thank you for continuing to work on this. I've been excited about Bot Colony since I got it last June. Such a great concept!
Thank you! The concept of characters that understand what the player says is unique and innovative, but may be ahead of its time. Not many people get excited about it (or know that Bot Colony game exists). I'll admit to inadequate marketing, problems with immature tech (and speech to text), an inexperienced dev team, etc - many factors that I did not foresee in 2007, when we started work on Bot Colony. Given the opportunity, I would have done a few things differently - but now it's too late.
I realize that some of you paid money to buy a game that will soon be impossible to play - because it requires online servers that cost money to keep running, while no money comes in.
I have advised Steam of the situation, and Bot Colony is no longer available to buy - in order to minimize the fallout when it will stop. We will keep the servers running until March 15, to give you enough time to enjoy the game that you bought. I hope you'll consider this fair. I'm offering to publish another video that tells the entire Bot Colony story - all the 12 episodes of Bot Colony, if you really care to see it (let's say if at least 10 people ask for it ). If anybody wants to buy Bot Colony knowing it will die March 15 - talk to Steam, and maybe they will sell it to you. For those who want to know the story that inspired Bot Colony and understand what robots could do in the near future, the Bot Colony novel is available here http://www.amazon.ca/Bot-Colony-Novel-Present-Future-ebook/dp/B00K7O6BSE
To our few fans, thank you for supporting us and for playing our game!
Major Update Launched today ( + Episode 3 Prototype Video)
With this update, Bot Colony offers more than commanding robots – the focus of the previous version. Now, you can investigate using your own words to discover the key information you need to advance in the game. To do this, we made a major improvement to our Natural Language Understanding pipeline – temporal Question Answering. We’ve always wanted to bring you a credible experience of speaking with machines. We hope that this update delivers on this vision.
We went to great lengths to reduce the percentage of IDK (I don’t know) answers. The key idea was to put in place a principle guiding what a robot should know: a robot should be able to answer questions about events that he experienced first-hand. Jimmy is now able to tell you how he came to know any item of information you’re uncovering (how do you know that?) – he’ll answer I saw it, or I heard it. To offer a completely credible experience, we’ve actually placed Jimmy and the characters in the scene, and then enacted the situations Jimmy observed using our text-to-animation technology: we moved the characters around like in a theatre performance and created a textual database out of their movements and utterances, that COULD have been used to drive our text-to-animation technology IF 1) we had IK for all the animations you’ll see (slapping, hugging, playing with toys, etc.) 2) animation quality was good enough. Since 1) and 2) are not yet there, the cut scenes were done by animators – but in principle they could have been generated just by commanding the characters to do what they do. The distances from Jimmy to another character he observes and the angles were logged during this enactment process, and you can query them. We’ve done this for 48 hours for the Intruder house. In the future, we may extend this to Arrival and other levels. By placing Jimmy in the situations he actually witnesses and recording what he saw and heard, we get more credible, complete and coherent information about what happened (Jimmy would be able to tell you who he saw at 11:30:41 on Thursday 27 August 2021, and that the distance was 10.3 m). This approach provides better coverage for question answering and reduces IDK – previously, Jimmy’s fact base had to be written manually. This is why this update took several months to release.
We plan to continue investing in making our robots more ‘intelligent’, and enabling you to get to information irrespective of the words you’re using – this is what is unique about the Bot Colony game experience, that no other game offers. We appreciate any feedback about things that didn’t work as you expected.
Finally, with this Update we’re also releasing a video of Riot (Landfall Episode 3) Prototype. In the Riot Humanity Test, you converse with robots about what emotions a certain situation is likely to trigger – their goal is to learn this to be able to able to react like a person ( simulate a person’s emotional behaviour through quasi-emotions, as described in the Bot Colony novel). All done through conversation, this is truly genre-defining gameplay!
UPDATE FEATURES AND HOW TO PLAY
• Robot Visual Memories (14 robot recorded video segments for you to discover)
The software now supports the basic investigative questions that you need to get to RVM. There is a time-context set by default for questions, so you don’t have to always specify the time and date (though you’ll need to specify it when you want precision). Here are some useful questions you can ask:
Who is X?
What do you know about X?
How do you know that?
When did you first/last see X?
What did X do at HH:MM on Day? (example: What did Ayame do at 20:15 on Thursday?) What did Ayame do then? What did Masaya do next? What did X do before that?
What happened then? What happened before/after that? What happened at HH:MM on (day of week)? (What happened at 11:30 on 26/08/2021?) – this will work even if after/before don’t return more facts because Jimmy doesn’t look back/forward far enough.
What happened to X? What happened to X at (time) on (date)/(day of week)?
When did X arrive/enter/leave the house? When did you arrive? Where did X go after that?
What did X say at (time) on (day)? What did X say before/after that?
Where was X at (time) on (day of the week)/date?
Other questions that will help you find out more about the characters, like :
What does X like? When did X play? When did Y speak on the phone?
• Dialogue history is accessible through F10. You may want to leave this window open during RVM gameplay. Use the middle mouse button to scroll through the dialogue text, use Up Arrow to repeat a previous question and hit Enter.
• We’ve added a Garage scene (the player’s home location) to better anchor the game story – your avatar starts here in 3rd person before you get the mission. This way, you actually see your own character before playing the ninja in Intruder.
• Male and female character selection is available before Arrival and you can play using a name you choose (we had to do that before Arrival because we can’t re-shoot Ogata, the Japanese actor proposing the mission to Jeff Philips).
• The game now has Save and Continue. This is critical to be able to re-play the RVM mission in different ways, WITHOUT having to worry about putting objects back in place. You do that once, Save the game, and then Continue from there in each new session.
• Jimmy now has lip synch.
• The align command is improved (and like on a ship or plane, red is on your left side and green on your right when you face forward).
• When you look through Jimmy’s eyes, you can control his head – useful to look inside cabinets, after Jimmy opens the door and blocks your view. Or to see inside the treasure chest.
KNOWN ISSUES
• Speech-to-text did NOT improve in this version, and we recommend that you type. If the game sells a decent number of units, we’ll be able to re-bundle Dragon NaturallySpeaking to offer you better performance (Dragon 11 is still supported, and early customers can still use it).
• Why questions are not supported – Jimmy doesn’t know why people do things.
• You may get an “I’m thinking…” message, when Jimmy needs more time to react to what you’ve asked him. Please bear with him.
• (Not a bug) Your investigation takes time, and you’re playing close to end of Friday 27 August 2021. Until Friday is over –ie, going past midnight Agrihan time, just saying ‘on Friday’ will mean Friday 20 August 2021. After midnight, it will mean Friday 27 August 2021. This CAN make a big difference in the answers you get.
THE GAME FOR GEEKS
With this update, Bot Colony has a lot to offer. To encourage more people to experience the game and especially this new update, we’ve decided to price the game at $9.95.
For non-native English speaker, Bot Colony is a fun way to improve your English. While Bot Colony appeals to many players, it looks like it has a special appeal to tech geeks and fans of sci-fi – please help us by mentioning our game to friends with these interests and Recommend-ing us on Steam.
Text-to-Animation video
https://www.dropbox.com/s/pai63dfy04530gx/Text-to-animation.mp4
WHY THIS MATTERS?
In Bot Colony, you can command robots to do things. What if we made this available as a tool in which you could import your own characters, a setting of your choice, and you could command the characters using a text script? (as you do in Intruder). Add special effects and editing tools, and you'd have what it takes to produce your own CG videos. With appropriate controls to synchronize character behaviour and speech, you could also make your own games (so, a modding tool). The key idea is to create animation through language, which opens this up to anyone (today, it's only animators).
A new medium of creative expression. Thoughts?
Bot Colony NLU technology - featured post in Gamasutra
http://www.gamasutra.com/blogs/EugeneJoseph/20140626/219765/Natural_Language_Understanding_and_TexttoAnimation_in_Bot_Colony.php
Twenty Questions free browser game available on www.botcolony.com
Jimmy will try to learn from you about why people do certain things. We plan to use the content that your provide to improve Jimmy's knowledge about people and every day life. Going forward, we hope that one day Jimmy will be smart enough so you'll consider adopting him as your VPA (Very Personal Assistant). Jimmy will get to know you, help you keep track of your daily and weekly priorities, and converse with you about what you need to do to achieve your objectives. We hope that Jimmy's language understanding will compare favorably with Apple's Siri or Microsoft's Cortana- if you're using either of them, we're keen to hear what you think.
Jimmy already remembers whatever you tell him. Experiment by telling him some facts, and then ask back and you'll see. Right now this works in the same sessions. We're working to make that persistent between sessions.
We'll put in place community features - there will be an AI Coach Ladder (games played, player wins against Jimmy, concepts taught, photos of objects uploaded, etc.)
Finally, please keep in mind this is an early Alpha of 20Q (even less, it's not yet function complete, but you can see where it is going). For now, you'll need to go to www.botcolony.com to play. We plan to make it available on Steam in the future - there's some work to port the client. If you find it interesting, please pass the link to a friend. As always, we need your feedback !