I've been continuing to try my best to clock in some hours toward fleshing out the Rhythm Quest level editor! Here's some quick demos of what is now possible in the editor:
More Tools
The level editor has a LOT of functionality that needs to be built out, so a lot of the time was spent just toward implementing additional tools, which you can see in the demos above. A no-so-short list of what I ended up implementing tool-wise:
"Insert Floor" tool to insert blank sections of track
"Delete Floor" tool to delete sections of track + included obstacles
"Jump" tool which inserts either ground jumps or air jumps
When inserting or deleting jumps, the other obstacles adjust (e.g. ground jump becomes air jump)
Visualization for jump arcs (also provides something to click to delete jumps)
"Flight Path" tool lets you click and drag to insert flight paths
"Ghost Enemy" tool for purple multi-hit enemies
"Edit Ramp" tool for adjusting the slope of ground / flight paths
"Checkpoint" tool for adding checkpoints
"Spike Enemy" tool for adding spike enemies
"Scroll Speed" tool for adjusting the relative scroll speed of each section
"Water/Speed Zone" tools let you click and drag to insert those zones
Character Preview
Something cool you'll notice in the gifs is an animated character preview that goes through the level. It's not too fancy, as it doesn't actually interact with any of the obstacles, but it's a fun little visual preview that didn't require a ton of effort to make happen.
During level generation I actually already calculate the entire player path through the level -- this is needed to understand where to place obstacles such as coins, flying enemies, and the like. (This is actually just one of several such calculated paths; there's another one for the camera, for instance) You can see that in this debug view in red here:
Since I already have this path on hand, it was simple to just create a preview sprite that traced along the same path at the correct speed. The only other work I needed to do was to trigger the appropriate animations based on the obstacle timings. There's no collision detection or anything like that, so it's very simple!
Tooltips and Notifications
Another thing you may have noticed is that I've added tooltips for the different buttons in the editor! Right now most of them are on a big palette on the left (with some additional ones on the bottom-right), and they're just icons, so I thought it would be nice to show a little tooltip when you over over each button:
Along with that, I also implemented a notification system at the top of the screen! This not only gives you a little more context for how to use each tool (some of them involve dragging, others just require clicking), but also displays information on actions such as undo / redo history:
I wanted this to look nice, so it supports the ability to either show multiple notifications at once (automatically scrolling them as they fade out), or replace an existing notification. The way this works is that there are different "slots" for notifications, so for example if there's already a tool selection notification showing, it'll just replace that existing one instead of showing a brand new one.
Menu Hookup
I also took a second to make the level editor accessible from the main menu:
Right now the "Custom Levels" menu is empty otherwise, but eventually there will need to be some way to import / browse existing levels (more work for later...).
As a side note, the buttons in the main menu now adjust their height dynamically instead of always being the same. This is because the main menu actually has more or less buttons depending on a whole slew of factors:
The "Quit" button isn't shown on web or mobile builds
The "Wishlist" button is only shown on demo builds
The new "Custom Levels" button may also have some restrictions (?) (TBD)
I don't know why I didn't do this earlier, but it was simple enough to set up with Unity's layout groups. I still wanted the "Start Game" button to be a little larger than the others, but I was able to set that up using custom LayoutElement components, so now that one is 125% the size of all the others, and it all happens automatically. Yay!
Waveform Rendering
I also ended up taking a little detour figuring out how to take a music file and calculate + render a waveform into a texture to display on the screen:
This task involves a surprising amount of technical finesse! A 1.5-minute song has some 4 million stereo audio samples, so obviously trying to plot and process all of that data is a little tricky to do in a performant and sensible way. Trying to draw a line between each of the 4 million points is pretty futile, so I didn't even bother doing that.
Instead, a common approach is to separate the audio samples into chunks -- in this case, one chunk for each column of the final output texture. Then for each audio chunk we can simply take the min and max signal amplitude for all of those samples and draw a line representing the magnitude of that. (you could also use other metrics, such as the average signal magnitude)
Because you're processing 4 million samples, this works OK, but is still a little slow. The other problem is how to actually draw all of the lines / pixels into the resulting texture in a way that's more efficient than simply calling Texture2D.SetPixel(...) thousands of thousands of times.
This is a rare case where I actually dug into the technical details of how to optimize the performance here -- luckily, there's a Unity blog post from earlier this year that describes some details of how to write to textures efficiently, and there's a link provided to some sample code that leverages the parallel job system and burst compiler to speed that up. It seems a little bit black-magicky, but it did the trick and I'm able to generate that texture on the fly without a noticeable hitch in framerate (woohoo!).
Right now since I'm just testing, the waveform appears as an ominous black-and-red texture behind the stage (haha), but eventually I hope to integrate this display into some sort of UI (?) that will help you tune the BPM and beat offset of the audio that you load into the editor. In case you're wondering, the texture is red-and-black because I'm using the lightweight one-byte-per-pixel R8 texture format (the same one I talked about in my backdrop optimizations post).
Next Steps
Despite all of the good work that I've been able to accomplish, there's still no shortage of work needed in order to bring the level editor into a fully-functioning state (not to mention a slew of extra quality-of-life features that I've thought of already). Chief among those is the menu interface for adjusting song timing properties (BPM/beat offset), which is why I started looking into the waveform rendering tech...but, there's also things like export/import functionality, backdrops, color palettes, particle effects, (the list goes on...). Hopefully I'll have even more to show off the next time I write an update!
Devlog 57 - Level Editor, Scoring Rework
I've been continuing to try my best to clock in some hours toward fleshing out the Rhythm Quest level editor! Here's some quick demos of what is now possible in the editor:
More Tools
The level editor has a LOT of functionality that needs to be built out, so a lot of the time was spent just toward implementing additional tools, which you can see in the demos above. A no-so-short list of what I ended up implementing tool-wise:
"Insert Floor" tool to insert blank sections of track
"Delete Floor" tool to delete sections of track + included obstacles
"Jump" tool which inserts either ground jumps or air jumps
When inserting or deleting jumps, the other obstacles adjust (e.g. ground jump becomes air jump)
Visualization for jump arcs (also provides something to click to delete jumps)
"Flight Path" tool lets you click and drag to insert flight paths
"Ghost Enemy" tool for purple multi-hit enemies
"Edit Ramp" tool for adjusting the slope of ground / flight paths
"Checkpoint" tool for adding checkpoints
"Spike Enemy" tool for adding spike enemies
"Scroll Speed" tool for adjusting the relative scroll speed of each section
"Water/Speed Zone" tools let you click and drag to insert those zones
Character Preview
Something cool you'll notice in the gifs is an animated character preview that goes through the level. It's not too fancy, as it doesn't actually interact with any of the obstacles, but it's a fun little visual preview that didn't require a ton of effort to make happen.
During level generation I actually already calculate the entire player path through the level -- this is needed to understand where to place obstacles such as coins, flying enemies, and the like. (This is actually just one of several such calculated paths; there's another one for the camera, for instance) You can see that in this debug view in red here:
Since I already have this path on hand, it was simple to just create a preview sprite that traced along the same path at the correct speed. The only other work I needed to do was to trigger the appropriate animations based on the obstacle timings. There's no collision detection or anything like that, so it's very simple!
Tooltips and Notifications
Another thing you may have noticed is that I've added tooltips for the different buttons in the editor! Right now most of them are on a big palette on the left (with some additional ones on the bottom-right), and they're just icons, so I thought it would be nice to show a little tooltip when you over over each button:
Along with that, I also implemented a notification system at the top of the screen! This not only gives you a little more context for how to use each tool (some of them involve dragging, others just require clicking), but also displays information on actions such as undo / redo history:
I wanted this to look nice, so it supports the ability to either show multiple notifications at once (automatically scrolling them as they fade out), or replace an existing notification. The way this works is that there are different "slots" for notifications, so for example if there's already a tool selection notification showing, it'll just replace that existing one instead of showing a brand new one.
Menu Hookup
I also took a second to make the level editor accessible from the main menu:
Right now the "Custom Levels" menu is empty otherwise, but eventually there will need to be some way to import / browse existing levels (more work for later...).
As a side note, the buttons in the main menu now adjust their height dynamically instead of always being the same. This is because the main menu actually has more or less buttons depending on a whole slew of factors:
The "Quit" button isn't shown on web or mobile builds
The "Wishlist" button is only shown on demo builds
The new "Custom Levels" button may also have some restrictions (?) (TBD)
I don't know why I didn't do this earlier, but it was simple enough to set up with Unity's layout groups. I still wanted the "Start Game" button to be a little larger than the others, but I was able to set that up using custom LayoutElement components, so now that one is 125% the size of all the others, and it all happens automatically. Yay!
Waveform Rendering
I also ended up taking a little detour figuring out how to take a music file and calculate + render a waveform into a texture to display on the screen:
This task involves a surprising amount of technical finesse! A 1.5-minute song has some 4 million stereo audio samples, so obviously trying to plot and process all of that data is a little tricky to do in a performant and sensible way. Trying to draw a line between each of the 4 million points is pretty futile, so I didn't even bother doing that.
Instead, a common approach is to separate the audio samples into chunks -- in this case, one chunk for each column of the final output texture. Then for each audio chunk we can simply take the min and max signal amplitude for all of those samples and draw a line representing the magnitude of that. (you could also use other metrics, such as the average signal magnitude)
Because you're processing 4 million samples, this works OK, but is still a little slow. The other problem is how to actually draw all of the lines / pixels into the resulting texture in a way that's more efficient than simply calling Texture2D.SetPixel(...) thousands of thousands of times.
This is a rare case where I actually dug into the technical details of how to optimize the performance here -- luckily, there's a Unity blog post from earlier this year that describes some details of how to write to textures efficiently, and there's a link provided to some sample code that leverages the parallel job system and burst compiler to speed that up. It seems a little bit black-magicky, but it did the trick and I'm able to generate that texture on the fly without a noticeable hitch in framerate (woohoo!).
Right now since I'm just testing, the waveform appears as an ominous black-and-red texture behind the stage (haha), but eventually I hope to integrate this display into some sort of UI (?) that will help you tune the BPM and beat offset of the audio that you load into the editor. In case you're wondering, the texture is red-and-black because I'm using the lightweight one-byte-per-pixel R8 texture format (the same one I talked about in my backdrop optimizations post).
Next Steps
Despite all of the good work that I've been able to accomplish, there's still no shortage of work needed in order to bring the level editor into a fully-functioning state (not to mention a slew of extra quality-of-life features that I've thought of already). Chief among those is the menu interface for adjusting song timing properties (BPM/beat offset), which is why I started looking into the waveform rendering tech...but, there's also things like export/import functionality, backdrops, color palettes, particle effects, (the list goes on...). Hopefully I'll have even more to show off the next time I write an update!
Devlog 57 - Level Editor, Scoring Rework
I've been kinda radio-silent over the past month. The first part of that was due to simply not getting that much done, but in the latter half of the month I had a better (and more exciting) reason for foregoing updates: I've been hard at work building out the initial skeleton of the Rhythm Quest Level Editor!
Changing My Mind on the Level Editor
Those of you who have been following along for a while will probably know that I've always had plans to build out a Level Editor for Rhythm Quest, but that I had initially pegged it as a post-release feature -- something that I'd build after the release of the initial game. I wanted to make sure that I focused my efforts on bringing the main game to a finished state, even if it meant sacrificing some extra features that couldn't make the cut. Building a level editor is no small task and I wasn't even sure exactly how I would be able to do it. Would it be an entirely separate application? What would the interface look like? Would I have to refactor the entire level generation process again? What formats would levels be saved in? How would I load in external music files? How should I handle copyright disclaimers?
Despite my strong belief that Rhythm Quest levels work best when you have someone (like me) carefully crafting both music and charting together, and ensuring that the levels follow canonical Rhythm Quest charting conventions, I understand that the level editor is a popularly-request feature and could really help to bring excitement to the game in a multitude of ways. But the reason that I decided to change my mind and start working on the level editor now (instead of post-release) is much simpler: I simply got interested in building it.
On-Demand Level Generation
Like so many other challenges that I've come across in working on Rhythm Quest (like the new coin/medal rework, which I'll talk about later), the level editor conundrum was one of those things that sort of just sat in the back of my mind for a long time until I had finally devoted enough spare idle cycles to it and was beginning to have some ideas of how to actually get started working on it. There is of course something to be said for keeping feature creep down, but I've learned that "working on whatever I'm excited about" is usually a good approach for keeping me going.
Rhythm Quest levels are authored as strings of events. Here's how the charting for level 1-3 is written out, for example:
The different symbols here are a representation of different types of events. '#' represents a checkpoint, for example, while '1' is a basic enemy, and '^' is a normal jump. ('*' is shorthand for a jump followed by a flying enemy.) When the engine parses this string, it converts it into its respective sequence of timed events, so something like:
_events = { new EventData(0.0f, EventType.Checkpoint), new EventData(8.0f, EventType.NormalJump), new EventData(8.5f, EventType.SingleEnemy), new EventData(12.0f, EventType.NormalJump), new EventData(12.5f, EventType.SingleEnemy), ... };
This (along with other metadata about the level) then gets passed off to the level generation procedure, which is responsible for processing all of the events in order and building the actual level out of them. Normally this is all done ahead-of-time when I author the levels (in a process I call "baking" the levels), so the end level objects are saved into the scenes directly to optimize load time.
Now, the way that the (work in progress) level editor works is simply by maintaining a similar list of events that compose the level being edited, and re-generating the level again every time there's any change. It might seem terribly inefficient to keep rebuilding the level compared to just editing the resulting level objects directly, but there's a lot of reasons why it makes sense to do things this way. For example, changing a list of events is simply more efficient than having to worry about editing the actual level objects (moving floors around, etc). and I already have the code to do all of this, so I just have to worry about providing an interface to visualize these changes well.
Testing the Prototype
Part of the reason I wanted to dive into working on the level editor right away was simply because I was curious whether this approach would even be feasible at all. I was worried that re-generating the level at each change might be too slow, for example. So I created a quick editor scene and made a script to hold a list of events, populated with some test data. I could then invoke the level generation process at runtime from there...
...and have everything be totally broken. All of the objects in the game are all built assuming that if the game is running, the level is supposed to be playing. They also assume that a song is playing, that they can query for the current music time, that a player instance exists, etc. So I had to do a bunch of refactoring to handle this unplanned-for case where we have all of these level objects, but they're not actively updating because we're in the level editor.
One thing I wanted to shoot for was to be able to instantly jump from the level editor into playing the level, without having to go through any sort of scene transition or anything like that. So I needed to make sure the level editing scene also contained everything needed for the base game, including the player instance, the music controller, etc. I also wanted to see if I could successfully load in audio files specified by the user. Here's what all of that looks like in action:
After doing all of these refactors, I had a simple prototype and I could add in basic enemies or ground ramps by pressing a key on the keyboard. One of the first things I did after that was to see what the performance was like when I triggered level generation, especially after I added a ton of events and made the level longer. To my delight and surprise (especially because the full level baking process normally takes a bit), the performance was actually pretty acceptable! I was initially expecting to see like 1-2 second pauses once the level got longer, but it seemed like it was only a minor hiccup most of the time.
This is also without any sort of optimization -- of which there could be many. Besides just raw "cache things, do work ahead of time, make the code faster", there's also the fact that most events shouldn't require the =entire= level to be rebuilt. Yes, a change in the ground ramp early on does mean that the height of the rest of the level will change, but at least you can skip re-generating everything that came before that. And adding or removing enemies shouldn't require the entire rest of the level to change. If it came down to it, I could force you to work on only one checkpoint section at a time. But it looks like I don't have to worry about any of those optimizations (yet).
Input Timeline
There's going to be a lot of work for me to do in the upcoming weeks for implementing various tools so that the editor can actually provide enough functionality to create a full level -- both in terms of all of the actual level obstacles (water zones, flight paths, etc.), as well as the level metadata (music BPM, scroll speed, background palettes). One thing I did in the meantime was to implement what I'm calling an "input timeline" feature, where the expected inputs are displayed as colored diamonds in a separate track below the level. I added this mostly for use in the level editor, but I also made it function in-game in case you want to use it there:
The exact look of this will probably need to be adjusted (not very colorblind-friendly right now either), but this is a really useful view for the editor already, and will probably become even more important once I look into more advanced editing features (editing via keyboard or even a "record" style live play). One thing about this input timeline is that you can see just how boring of a game Rhythm Quest is in terms of the raw inputs. A big part of the appeal of the game (to me, at least) is parsing the platformer-like obstacles into rhythmic elements; if the chart is just laid out in front of you like this it's really not too interesting.
Scoring Rework
I did this a while ago but never wrote about it. Despite the fact that I've already tweaked the coin / respawn / medal scoring system a few times (at various points in time it's alternatively been based on respawns and coins), I've iterated on the system once again. I was never happy with how the medal thresholds felt both arbitrary and also not very visible, so I worked out a "progress bar"-style animation in the level end screen to show that off visually:
The thresholds are now straightforward and easy to remember based on the visual (25% = bronze, 50% = silver, 75% = gold, 100% = perfect). Previously you were awarded a bronze medal simply for completing a level, but I've changed that, so you'll just have no medal if you finished with less than 25% coins.
Along with this, I'm trying out a new system for coin loss amounts. Previously you always lost a fixed amount (5 coins) on every respawn, but this usually led to people either getting very close to perfect, or losing almost all of their coins on a particular section or two that they had trouble on, even if they performed very well through the rest of the song. I've always wanted something that scales more cleanly, like for example every time you respawn you lose 50% of your coins, but that by itself doesn't work well because it's extremely punishing for single mistakes that are made late in a level.
The way it works now is more complicated, but should hopefully be more "balanced" in terms of coin losses. The new system internally maintains two different coin numbers -- the coins that you have recently collected, and the coins that you have "banked". At every checkpoint, half of the coins you have on hand are put into the "bank" and can never be lost from then on. And at every respawn, half of your non-banked coins are lost. The idea is that this system rewards you for performing well, and can't fully "take away" that reward even if you mess up a lot afterwards. It's a bit obtuse in that it's a pretty hidden mechanic, but I like the simplicity of implementation and the fact that I'm not using some really random number like 5. We'll have to see how it works in practice, though!
That's going to do it for now. I'm trying my best to get the level editor off the ground...it's a lot of work, but also interesting and exciting since there's so many little systems that need to be written, for the first time! There's unfortunately a good chance that this will end up pushing back my launch date to 2024, but...I'm hoping you'll all agree that the custom levels that will come out of this will be worth the wait.
Rhythm Quest Demo v0.29.0 Released
The Rhythm Quest Demo has been updated to version 0.29.0! This patch fixes a bug with the latency calibration measurements (you may want to re-calibrate) and reworks the coin/medal system, along with other fixes and improvements.
Full changelog:
Version 0.29.0 - Fixed auto-calibration output values (was previously ~75% too low) - Reverted level medals to be based on coin count again - Changed coin thresholds for medals (25%, 50%, 75%) - Changed coin loss amount/mechanic to be much less punishing: 50% of non-banked coins are now "banked" on each checkpoint and can no longer be lost 50% of non-banked coins are lost with each respawn - Added new level end coin/medal display - Updated ghost teleport to change speed based on music speed mod - Updated input handling interactions to be more timing-accurate - Fixed an input bug triggered by having multiple inputs on the same frame - Fixed a bug causing the color invert fx on doublehit enemies to persist - Fixed a timeslicing issue that sometimes caused interactables to be skipped at high speeds - Try (again) to fix rendering issues at odd resolutions - Added Respawn Timing setting - Fixed ghost enemy trail effect to render consistently regardless of framerate/speed - Fixed screen refresh rate label not actually updating - Localization and UI updates - Fixed respawn count not resetting properly when checkpoints are disabled - Fixed the ability to finish a level and respawn simultaneously
Rhythm Quest Demo v0.28.0 Released
The Rhythm Quest Demo has been updated to version 0.28.0! This patch includes a multitude of extra settings and game mods, including the ability to play levels at a faster or slower speed!
Full changelog:
Version 0.28.0 - Fixed ramped floors being allowed with spike enemies - Renamed "Cheats" menu into "Game Mods" - Added assists/game mod settings to in-game menu - Made shop accessible from Extras menu - Added music speed game mod setting - Added timing window game mod setting - Added ghost helper game mod setting - Added frame queueing graphics setting - Added configurable bindings for gamepad controls - Keyboard bindings now default to entire left/right half of main keyboard area - Fixed rendering artifacts at odd resolutions - Added UI volume setting, separate from sfx volume, reduced UI sfx volume slightly - Change pixel font setting default based on language at runtime - Added low quality graphics toggle to help framerate for older devices - Added "Released Early" and "Released Late" text for hold presses - Added "screenreader" command line argument force start with screenreader prompt - Fixed too-fast scrolling before level 1-1 tutorial - Fixed fullscreen not working for devices that present resolutions in an unexpected order - Fixed vsync off behavior, especially for desktop platforms - Changed flashing effects to be time-based rather than frame-based, to support higher refresh rates - Settings are now saved when drilling into a submenu (not only when navigating back) - Progress is now saved (again) while auto play is enabled - Thickened some outlines on smooth text - Minor tweaks to jump logic (hopefully not breaking anything)
Version 0.27.0 - Added tracking of respawn counts to save data and level display - Localization and UI updates
Devlog 56 - Gamepad Rebinds, Odds and Ends
Despite what it might seem like, I've actually been working on quite a lot of different things for Rhythm Quest recently! Let's get right into it...
Game/Music Speed Mod
You can now change the speed of the music to make the game more or less difficult:
Some of you might be wondering why this took so long to come to the game, considering how this has actually been a debug-only feature for a while. Changing the speed/pitch of the music isn't actually very hard, the problem is getting all of the music synchronization (and respawn, and pause) logic to work properly in tandem with it.
I'm happy to report that this has been implemented properly now! It's definitely not as simple as it might seem...every time you change the setting in the pause menu, what I'd =like= to do is to immediately increase the speed of the currently-playing pause music loop, then adjust the audio timing variables accordingly. But there's no way to do that in a way that respect the audio synchronization perfectly, since the audio timings are all running separately.
Instead I have another copy of the pause music which is set to play at the new pitch/speed. Instead of trying to play that one immediately, I schedule it to start playing 0.2 seconds from now (scheduling sounds in advance is the only way to ensure that they trigger at a precise time on the audio timeline). Unfortunately, I can't schedule a "stop" command in the same way for the first version, so instead I simply wait for roughly 0.2 seconds and then do a flip where I set the volume of the first loop to 0 and the second one to 1 at the same time. Doing all of this lets me keep the beat synchronized with all of my timers correctly.
Of course, since the whole process does take a little bit of time (0.2 seconds), I also needed to implement a simple wait, in case you trigger the button twice in quick succession -- the second music speed change doesn't occur until the first one is finished.
Anyways, this will be available in the upcoming 0.28.0 patch, so players can either use lower speeds to practice / get through difficult sections of the game, OR use higher speeds to give themselves an extra challenge.
Timing Windows
I also have a brand new setting for modifying the lenience of the timing windows for obstacles:
This affects the collider sizes of the various obstacles in the game. Here's how that looks for a basic flying enemy:
Originally I had a crazy half-baked idea in my head that to implement "extra leniency" I would actually buffer inputs for you and delay them automatically to "auto-correct" your early inputs...and for late inputs, I would just "freeze" the character in place for a split second to give you extra time to hit the correct button. I realized, though, that this would make empty jumps and attacks (the ones you do for no reason) feel really sluggish and awkward. I could try and do some tricks to correct for it, but in the end I figured that modifying the sizing of the colliders was simpler and just as effective, while maintaining the tie between your input and the game reacting immediately.
One cool thing is that the setting works for jumps too, even though you wouldn't think they have a "collider" to modify. This is because every jump (even if it's just a normal one over a pit of spikes) has an invisible trigger object (this is what spawns the blue "spinning square" visual effect). As long as you jump while you're inside of that trigger zone, my jump logic will autocorrect the length of your jump to make you land in the appropriate spot. I also already implemented "coyote time" jumps that will allow you to jump even after running off a cliff (as long as you're still within the correct trigger area), so it all just works. Here's that in action with the extra wide timing windows:
Ghost Helpers
I got to see and/or hear about a variety of play experiences when I did my last internal playtest. I won't lie: the design of Rhythm Quest is almost entirely driven by my own sensibilities, but it's still useful to see other perspectives so I can make small course-corrections and admissions when I feel necessary.
Interestingly (or maybe this shouldn't really be surprising), different people seemed to struggle more or less with different aspects of the game -- for some, the water zones really threw off their ability to read the charts, while other people mentioned the ghost enemies being difficult to read since they felt like they needed to react to each of the new positions.
For people who struggle with the ghost enemies, I've added a new helper option that will display guidelines for the second and third positions of the ghosts:
This does, of course, take away from the entire conceit of the ghost enemies to begin with (with the helpers, it's no different than three red enemies), but I really don't mind adding these sorts of modifiers when the implementation is so simple (and I'm not busy working on other things). You can play Rhythm Quest however you want!
Performance Settings
I already did a bunch of work on performance optimizations in a previous stint of work, but I'm happy to report that I've also made the rendering of the water sections more performant: previously, the render textures used to implement the "wavy" water shader were way bigger than they needed to be...now they don't extend past the height of the screen, and dynamically shift based on the camera to be more efficient.
However, I also wanted to give a nod to some lower-end devices, and ended up adding a graphics quality toggle that will disable the fancy water shader entirely, as well as cut out some of the translucent backdrop layers (that I pick by hand) in an attempt to make rendering more performant. I also added another engine-level setting for controlling frame queueing. Those all live on a new sub-settings page that comes complete with a quick-and-dirty FPS counter and some water zones in the background so you can see the effects of the settings live:
Other Stuff
Some miscellaneous other stuff was added too, like a "screenreader" command-line flag that will force the game to start in screenreader mode even if it doesn't auto-detect that one is enabled.
In an attempt to add a little more discoverability to the shop menu, I also made that accessible from the "Extras" menu in addition to the normal place in the level select screen. I also renamed "Cheats" as "Game Mods" since I'm throwing in all of the timing and visual modifications that I've added there. Most of those are also accessible via the in-game menu, which can be helpful if you need a temporary assist for a tough checkpoint that you're struggling on.
I think that about covers it for what I've been working on lately!
Devlog 55 - Gamepad Rebinds, Odds and Ends
I'm continuing my break from working on world 6 levels for now. I've sent out the current build to some internal beta testers to get some feedback on the difficulty scaling and reception to the newer mechanics, so I want to give myself a chance to let that feedback come in and stew on it for a bit before I continue on with the last 4 levels of the game.
In the meantime, I've been trying to tackle some improvements and fixes that have been laying around in my backlog for a while...
Gamepad Rebinds
This one has been desired (and requested) a long while ago, but I kept on putting it off because I wasn't sure exactly how I wanted to handle it.
I already had control rebindings working just fine for keyboard controls, which have a single key assigned to each action:
The problem with gamepad bindings is that by default the gamepad controls have many different bindings: to jump you can use the d-pad, the left analog stick, the south or east face buttons, or even the left shoulder button.
I was sort of at a loss for how to deal with this, both in terms of the UI (how to show the combined default bindings?) and in terms of implementation (how to override the entire set of bindings at once?).
Like many other tricky problems I've run across in Rhythm Quest, letting it sit in the back of my head for a while allowed me to come up with a different approach:
Gamepad and keyboard bindings now each have their own standalone submenu (not available on platforms where they don't apply). More importantly, there's an individual setting that toggles between a "default binds" set and a "custom binds" set. The default binding set features multiple binds, whereas the custom binding set only has two (that can be overriden by the user). This elegantly (?) solves the issue I mentioned above.
This also lets me illustrate the controls in a hand-drawn diagram, something that's probably easier to parse than "Jump: DPad, Left Stick, Left Shoulder, A, B, ..."
Using the same system, I'm even able to detect whether a (supported) gamepad is plugged in at all, and dynamically update the screen accordingly:
I adopted the same tech for the keyboard bindings screen as well (had a bit of fun trying to draw a keyboard layout):
You'll notice that I decided to also expand the default bindings to just encompass the entire left/right half of the main keyboard keys. Unity does a reasonably good job (?) of detecting keys based on physical location, so this should work even if you use a nonstandard key layout like I do. I'm not sure what will happen for non-ANSI physical layouts, but I'm assuming the custom binding system will suffice for any odd edge cases.
For now I'm providing two custom binding slots for each action (an improvement over before where you could only use one key), in case you want to alternate keys for faster sections.
As usual, there's a ton of silly little details that need to be handled with input rebindings, and as usual, Unity provides just enough functionality to be helpful, but also forces you to work with a ton of abstractions like "InputControlPaths", "InputActions", and "ControlSchemes" that end up making your head spin when you think about them too much. You need to, for example, make sure that a rebinding can be cancelled via either Gamepad OR Keyboard input (the input system by default only allows you to define a single cancellation binding...)...
Rendering Artifacts
This is a really silly one, the kind of thing that you'd never imagine would be an issue, but somehow it is. Rendering the game to a width or height that's an odd number (e.g. 1013x533) causes weird visual artifacts:
This is caused by camera scaling and such -- here, the resolution is 501x301 and the game has decided to render the pixels at 2x, which means the base resolution is 250.50x150.50, which doesn't work out too nicely.
I tried to address this before by automatically resizing the game window and forcing it to be a multiple of two, but that didn't work too well. My new solution is to handle the rendering properly by shifting the camera by a similar fractional amount, so here we simply shift the camera over by a half pixel and fortunately that works to fix things.
Released Early/Late
Suggested by one of my playtesters -- the "Too Early / Too Late" text for holds is now more specific in calling out "Released Early / Released Late". A super easy fix that hopefully helps clarity a tiny bit:
I'm glad I got around to some of these improvements and fixes (which should be coming to the demo soon), but I feel like I've only just scratched the surface of the work that needs to be done. Even for the gamepad rebinding system, I still need to test how it works on Switch / for other types of gamepads, and could even stand to draw different graphics (especially for the Switch joycons). There's also some tweaks that I'm going to be trying to look at after seeing how playtesters fared with the current build...
The year is about halfway over and unfortunately my progress hasn't been super great -- I've only managed to finish off 6 levels in that time, plus some optimization work/etc. Of course, I had some real life stuff happen that drew my attention away, but that's also sort of true in the upcoming months as I help mentor for a video game tournament. That "end of 2023" date is starting to feel really scary when I think about it...
Rhythm Quest Demo v0.26.5 Released
The Rhythm Quest Demo has been updated to version 0.26.5! This patch includes some hefty optimizations, which should help improve performance, particularly on lower-end machines.
Full changelog:
Version 0.26.5 - Reworked texture encoding for memory and performance benefits - Various other performance optimizations - Fixed an issue causing minor vertical blurring on WebGL builds - Add better transition for Furball attack -> jump - Fixed attack animation being cancelled on fly start - Fixed menu bug where foreground level backdrops sometimes failed to fade in - Minor UI fixes
Devlog 54 - Backdrop Optimizations
Somewhat unexpectedly, I took a break from working on levels this month to focus instead of **performance and memory optimizations**. This was brought on by the fact that I made some release builds for the first time in a while and found that my iOS build crashed on startup because it was running out of memory loading the main menu!
The main culprit? These huge backdrop texture atlases...(this one is 64 MB!)...
The Problem
Your first thought upon seeing these atlases is that they're really wasteful. Why is there so much empty space in the upper-right? Well, that one is because the texture atlases need to be even powers of 2 in dimensions (1024, 2046, 4096). I could, of course, have each layer be separate, without packing them into a single atlas, but then I'd lose all the performance benefits of being able to batch the draw calls for all of the background layers together.
The better question is why does each backdrop layer have so much vertical padding? Well, I didn't want to make any assumptions about the player's aspect ratio, resolution, or zoom settings, so the easiest way for me to solve that was to just author all of my backdrop layers with loads of vertical leeway, so that they'll always be fully in view.
Each separate layer is exported at 500x1200 pixels (very tall!), and then tiled horizontally by the game. Some of the levels have upwards of 10 or 15 separate backdrop layers, so that's quite a lot of pixels...
Texture Encoding
The first thing I wanted to do was see if I could just store the textures more efficiently without changing anything about my authoring workflow. You may have noticed that the texture atlases are all grayscale (no color). This is a change I made a long time ago, back when I decided to use a palette shader for the backdrops. Essentially, I only really need to represent indices into my color palette (currently, one of 11 colors), so during my export I just use grayscale colors that the pixel/fragment shader can read and then interpret as color 0, color 1, etc. I also sometimes have partial transparency, so the alpha value is also important.
However, the textures are still encoded as 32-bit RGBA, which means 8 bits are assigned to each of the red, green, blue, and alpha channels! That's pretty wasteful, so I wanted to look into whether Unity supports other lossless texture formats (across multiple platforms). It does, in fact you can actually use the "R 8" texture format, which exclusively encodes a red channel (nothing else!), and only uses 8 bits per pixel (25% of what I was currently using!).
That seemed perfect, as really all I needed was grayscale values anyways. The one problem was that I still needed to store alpha values to handle partial transparency. Could I somehow pack both the color index, and the alpha information, into 8 bits?
Since I only have 11 different colors in my color index, 4 bits is enough to encode that (2^4 = 16). That would leave the other 4 bits to store alpha information, which would mean I could have 16 different possible alpha values. That's more than enough for my purposes, so I went ahead with this strategy of using 4 bits for color encoding and the other 4 bits for alpha values:
To get this all working, I needed to first write a python script to take all of my original backdrop exports and encode them into an 8-bit red channel like you see above. Then I needed to modify my palette shader to do the reverse: take the 8-bit encoding and parse it into a color index and an alpha value.
After a bunch of shader math debugging and fussing around with bit arithmetic, it was all working (everything looked the same as before) and the iOS build was no longer crashing. Hooray!
Texture Cropping
We can still do better, of course. The next step was to see if I could get rid of all of the extra padding on the top and bottom of many of these images. Take this cloud layer for instance:
Ideally we could only store the actual texture data that really matters (the middle section). The top half is all transparent, so we can just discard that, and then for the bottom half we can just "clamp" the texture lookup so that the bottom-most opaque row is essentially repeated indefinitely.
Doing the crop itself is simple enough -- I just modify my python image-processing script to analyze the rows of the image and trim it accordingly. We end up with this nice cropped version of the image:
The trickier part is that we now need to render this in the same way as the original texture. There are a couple of problems with this...
First, the new origin/center point of the sprite is different than before, since we trimmed an unequal amount of rows from the top and bottom, so it's going to be offset from where it was supposed to be drawn. To fix this, I added processing to my script to keep track of how much the new cropped sprite is offset by. I also track some other important metadata, such as whether the top or bottom sections (or both) should be repeated transparency, or a repeated opaque row. Then I output that all to a C# file that I can read in:
My backdrop tiling script is responsible for taking the stored offset metadata and shifting the center position of the rendered sprite accordingly.
The second issue is that while Unity supports texture coordinate clamping, there's no way to do that when the sprite in question is one of many sprites packed into a texture atlas! Unity's sprite renderer only handles tiling in a very specific way, which no longer applied to what I wanted to do, so I had to modify my fragment shader to handle the texture clamping part.
In order to do this texture clamping correctly, I also needed my fragment shader to understand what UV texture coordinates it was supposed to be working with inside the texture atlas. Normally the fragment shader is completely oblivious of this -- the Sprite renderer is responsible for handing it a set of UVs to render and then the shader just does the texture lookups blindly.
It also turns out that you don't actually have access to the sprite UV metadata from within your fragment shader =/. So I needed to pass those into the shader, =and= I couldn't use uniform variables since that would break batching. Luckily, Unity happens to expose a SpriteDataAccessExtensions class which allows you to write to the UV texture coordinates of the sprite mesh used by a sprite renderer internally.
In addition to allowing you to modify the main UVs, it also lets you set additional texture coordinates on the mesh (TexCoord1, TexCoord2, TexCoord3, etc.). I used those to pass extra data to the vertex shader -- and then through to the fragment shader -- including the sprite UVs from the texture atlas.
This took a lot more debugging to get right, but at the end of all that, it was working! Here's the new version of the texture atlas from before (in all its red-channel glory), which is 1024x1024 instead of 4096x4096, and 1 MB instead of 64 MB!
Alleviating Overdraw
Rhythm Quest isn't really a performance-intensive game, so it runs fine on most systems. That said, there are a couple of areas where it can get into performance issues on lower-end devices (surprisingly, the Nintendo Switch is the main culprit of this so far).
One major performance bottleneck involves overdraw, which is a term used to describe when pixels need to be rendered multiple times -- typically an issue when there are many different transparent / not-fully-opaque objects rendered in the same scene (*cough* backdrop layers *cough*).
Unlike in a generic 3d scene (where we would try to render things from front-to-back, to minimize overdraw), for our backdrop layers we need to render things from back-to-front in order to handle transparency correctly:
Unfortunately, this results parts of the screen being rendered to many times over and over again, particularly the lower areas (all of those overlapping cloud layers...). The good news is that the cropping we did above already does some work to alleviate this a bit. Before, the large transparent portions of backdrops would still need to go through texture lookups and be rendered via the fragment shader, even though they were completely transparent (i.e. didn't affect the output). But now, we've cropped those areas out of the sprite rendering entirely, so they aren't a concern.
We can still do a little more optimization, though, for opaque backdrop sections! Take this layering of opaque cloud layers from level 2-5 as an example:
There's a lot of overdraw happening on the bottom sections of the screen. What if we were smart about this and kept track of which portions of the screen are being completely covered by each layer, front-to-back? That would let us render smaller screen sections for all of the back layers:
We can handle this by having our image processing script store some additional metadata (the "OpaqueBelow" and "OpaqueAbove" fields) so we know at which point a background layer obscures everything above or below it. We then need to modify the backdrop script to adjust the drawing rect and UVs accordingly (easier said than done...)...
The end result of all of this is...that everything looks exactly the same as before...
But! It's significantly more efficient both in terms of memory usage and rendering time. I'll have to patch the existing demo builds with this optimization at some point, but the Switch build is already showing some improvements, which is nice.
We're not completely done with performance though, as right now the rendering of the water sections are also quite inefficient! I may try to tackle that next...
Devlog 53 - Level 6-1
I'm continuing to just roll ahead with levels! It's funny, I feel like there was a long period of time when working on new levels and thinking about the mechanics felt intimidating, so I would just procrastinate on it and work on other miscellaneous things. But now I think it's the opposite (probably partly because all of my mechanics are known now), where I've gotten into the habit of just working on only levels. It's good though, the levels are something that need to be done 100%.
Anyways, I went straight ahead and finished up the first level in world 6, level 6-1!
Speed Zones
World 6 introduces one new mechanic, the red "speed zones" that increase scroll speed and change up the rhythmic meter into triplet patterns (quarter note triplets) temporarily:
As with some of my other mechanics, this might get mixed initial reactions from players (or at least, that's the expectation I'm setting up for myself...). For people who don't "get" triplet meter, it might seem sort of like an arbitrary changeup/speedup that's hard to react to. I experimented with having a sort of 2-beat "lead-in" to prep you for the new meter, but I was pretty unhappy with how that sounded (messy...) so I took it out. (Maybe that'll be an optional toggle someday?)
For now I'm just trying to give the player some easy speed zones at first so that they can listen to and get used to the rhythm, before I throw actual quarter-note triplets at them:
You might not have noticed it until I pointed it out (now it'll stick out like a sore thumb...), but none of the speed zones have any height ramps -- they're all completely flat. I couldn't really get the "conveyor belt" graphic to look reasonably good at an angle, so I just decided to add that as a restriction (the level generator will probably be really confused if you try to add ramps in the middle of it). I'm totally ok with that though, it makes them simple to read...and I actually have the same restriction for spike enemies (they can travel across ramps, but the actual jump needs to be on flat ground), so it's not really a new thing. I guess technically I can support height changes in the form of air jump combos and flight paths, but those haven't come up yet.
Visual Identity
This one was easy since I had already been thinking for a long time to do an outer space theme for world 6 (maybe sort of a trope to have the final area be space-themed?). One of the worries here is that all of the level backdrops are just going to look similar since they'll all just be dark skies with stars, but hopefully I can make them a little bit distinct by experimenting with different foreground elements and such.
For this level I went with sort of a "spiral galaxy"-type drawing with a bright orb in the middle. In hindsight, I probably could have drawn it bigger...but I guess this way it's more of a single element rather than filling most of the screen, which works too. It looks like there's all sorts of colors in there, but it's really just the 8-color palette, but with a bunch of translucent layers. It was actually quite fun to draw, as it felt like more of a painterly (impressionistic?) approach throwing blobs and dots of colors everywhere rather than the geometric shapes from world 5. You can also see that I'm making heavy use of the spraypaint tool for the first time here, particularly in the soft "nebula"-like patterns in the background.
As usual, I tried to add in some amount of variation in the color palette depending on the different sections of music. Here I switch to a completely black background color for the first "main" section of the song to up the contrast level a little bit:
I'm hopeful about this art style for world 6! Hopefully I'll be able to draw some nice backdrops by experimenting with this general direction. I was a bit worried at first since I feel like "space" art tends to not do well with such limited color palettes, but it's turning out fine with clever use of dithering-like effects and translucency.
Musical Identity
Unlike with world 5, I didn't do a whole ton of musical exploration before starting off on this level...I sort of just "winged it" and went with some rough ideas, seeing what came out of them. I knew I wanted to try playing around with whole-tone scale melodies, but I was also interested in exploring more varied bass sounds (maybe even dubstep-esque), as well as featuring prominent use of arpeggios and low-pass/high-pass filter automation.
Here's a snippet showcasing the "wub" bass featured in this track, as well as a triangle wave synth that plays a whole tone scale pattern. I dunno, somehow wobbly basses and triplet rhythms almost seems like a bit of a musical trope...
And here's a longer snippet of the main buildup in the song. I use a different (but still-prominent) bass here, and slowly open up the filter as it builds. As with world 5, I'm making heavy use of triangle-wave tom fills to accentuate the rhythmic changeups.
A new world also means a new level select theme! Here's a short video where you can hear that in action:
I had a few false starts on this one before I landed on the idea, but it sounds great! I love how the major IV -> minor iv progression works here. You can hopefully hear the low-pass filter automation on the chorded synth, as well as the reverbed short arpeggio pattern -- same ideas as in the level.
There's still a bunch more to explore with speed zones and how they combine with the other mechanics, which should be interesting to figure out over the course of these next 4 levels! I might have to tread a tricky balance since fast rhythms (e.g. double-hit enemies) are =really= fast in speed zones, so those will only be feasible if I take the overall tempo down a notch...
