AI War 2
Beta 3.744 Pathfinding
One of the most annoying issues lately has been "pathfinder errors," and it turns out they were a source of memory leaks, as well. And also just some general performance degradation, and over-use of RAM. I've done a massive refactor on it, and it now uses literally multiple orders of magnitude less data, which is great for everyone. It's not super well tested yet, but it seems to work from what I can tell so far.
There's a new issue where if you try to do the F7 dump prior to the first time you load into a game, and then load into a game, it locks up. No idea what that's about, but it's on my list for soon. The number of truly severe issues seems to be dropping, which is good, and at the same time we're getting into a position where performance is able to be better than ever before.
I also made a new... kind of complicated to understand, but easy to use... data structure called ThreadStaticWrapperedPooledItem. This funky thing lets us use the magic of the ThreadStatic while not having collections and other large portions of data orphaned on threads that are no longer using those items. Over time, I'll likely convert even more things to using this rather than just directly using ThreadStatic collections. It should be a notable improvement in memory usage, without any extra CPU cost during normal operations. This thing is one of the strangest pieces of code I've written, but despite that it is something I'm really pleased with because of just how effective it is.
Tom fixed a whole bunch of other errors in this release, including notifications not showing up properly in the prior version (that was an oops on my part). Lots more excellent bug reporting and narrowing-down of issues from Daniexpert, saving me and others a huge amount of time. Seriously, I'm so grateful to our testers, Daniexpert most of all recently.
Badger is continuing to improve the Elderlings in DLC3 for those testing that, and Zeus has made some very exciting further improvements to the new Orbital AI type in DLC3 (it's one of the most surreal AI types we've ever had, and an instant favorite for me).
Tom also added a number of quality of life improvements to various parts of the UI, which is very cool!
Tomorrow I'm hoping to knock out the strange "ships are showing when they should not be" bug and a number of other things. This pathfinder issue was always going to be a huge time sink, but I'm glad to have it done. And the new ThreadStaticWrapperedPooledItem is a new power tool that I can start strategically using anywhere else where we start seeming to have too many allocations over the course of many game loads.
More to come soon.
Enjoy!
Beta 3.743 Return Of The Imperial Spire
We're getting closer to having stability, but not there yet. Fallen Spire, on the other hand, seem to be pretty much returned to their former glory thanks to Tom, and the main things missing now are the final bolstering interface and then the new modular stuff.
NR SirLimbo continues to work magic on the technical front, adding many new options to how outguard can spawn things, which is quite exciting for modders as well as the DLC developers.
Brutal Lair is a little less brutal, thanks to Badger toning that down.
A lot of new cool things to test if you're in the DLC3 beta, related to Necromancer start options, the Elderlings being better and more interesting, and so on.
More bugfixes and hardening of code and other general refactors and improvements from me, and from Tom, and from Badger. Some balance tweaks and fixes from Zeus and CRCGamer. Daniexpert continues to be a super help with the debugging of things like serialization issues, etc.
It's coming along, and more is coming soon.
Enjoy!
Beta 3.742 "Classic Map" For DLC1
1. First off, there's a new map type for DLC1 (hey, wasn't that complete in early 2020?) based loosely on some of the favorite maps from AI War Classic. In general, we're going to be beefing up DLC1 and adding more to it over the next few months, then renaming it and slightly raising the price. Folks just don't seem sold on this DLC to the same degree that DLC2 is exciting them, and so we'd like to fix that so that it's truly a must-have. If you already have the DLC (or buy it now while it's still slightly cheaper), you're just getting this bonus stuff for free.
2. Lots of bugfixes, including many for the Fallen Spire. Huge thanks to Tom for most of these. The Fallen Spire is apparently pretty much fully playable again on the beta branch, although there's still some bits I need to finish to fully bring it home.
3. Badger's done a lot with the Necromancer faction for those folks testing DLC3, and feedback would be awesome. Not only that, but there's a new Elderlings faction that is related to Necromancers, or that you can use independently of them, and which are kind of roaming monsters in the galaxy. I'm really excited about this addition. DLC3 is really just getting kind of mind-blowingly large, and I'm really excited about it. That's one of several reasons we really needed to up the game on DLC1 so that it feels comparable to these other two giants.
4. Consulting with Tom, there's a new strategy for how I'm quarantining ships that have recently died, so that there's a lot less chance of random errors in the codebase related to UI elements or otherwise still referencing something that is in the process of being disassembled. The changes here solve a whole host of known bugs as well as a bunch of likely unknown ones. Tom did further refinement to fix more of them, with a lot of debug assisting from Daniexpert.
5. Speaking of Daniexpert, they've been invaluable for helping me find and fix more memory leaks and odd-allocations of RAM over the last few days. There are several notable fixes that really eliminate the most notable memory leaks that we've been able to find so far, and the game also runs more smoothly than ever. Whether there are any more memory leaks remaining is not yet clear, so please do let us know what you find!
There are a number of known exceptions that I haven't had a chance to look at yet, both reported on Steam and on mantis, but I'll be getting to those mainly on Tuesday. I've got some other things going on on Sunday and Monday that will cut into my work time a lot, but I'll be back at things on Tuesday.
More to come soon.
Enjoy!
Beta 3.741 Clogging Holes
1. On the subject of trying to hot-reload the xml files:
I've made some progress on that, but there's still more to do. There's some added functionality I need to add, apparently, before that's really going to work as well as I need it to. But it definitely got a lot closer today, and this took about the first half of the day to work on it. Right now if you try to hot reload xml, it will potentially lock up the game, but when you kill it and reload it, it will have what you want. I just need to fix the whole lock up part, but it's one step at a time.
2. On the subject of the memory leaks, which still exist:
Notice I'm not saying "the memory leak" anymore? I do suspect that there was mainly one big memory leak in the past, but at this point there are definitely multiple. Some of them are caused by the semantic changes I've made to how we use collections -- aka, in building the panopticon, I introduced new temporary memory leaks, but in exchange I gain the ability to find them and kill them, and also to improve performance in general in the future. I fixed at least six memory leaks today, none of which were the "big one" that is causing the most trouble.
Daniexpert was kind enough to provide a very useful core dump from that new feature, and by diffing that against an early-stage core dump of my own, this is how I'm not just finding the memory leaks, but also places where I can optimize capacity allocations. The side effect of the latter is that the game runs faster, and most notably that it's wicked fast to load savegames now. I mean, it's down from something like 1.5 seconds to 0.5 seconds; that hardly matters, but it does feel nice. And the overall ram usage being lowered, and the during-gameplay performance improvements that this also leads to, are the real win.
I'm currently going painstakingly through that one core dump, and just fixing things as I find them after careful analysis, rather than trying to find the big leak that I know is in there. Something has allocated 23 million objects when it really should be closer to three million based on another metric, and in that core dump is the answer to that question. When I find that, hopefully tomorrow or the next working day, that should be the end of the big memory leak (knock on wood). In the meantime, I don't want to leave any smaller memory leaks around just because I'm in a hurry, so I'm taking the time to do it right.
3. On the subject of the stalling background threads:
This is really intermittent and frustrating, and requires you to more or less restart the game to get it to work properly after it happens. It only seems to happen after having loaded multiple savegames in one run of the game. While chatting with Daniexpert about the data he was able to show me from it happening to him again today, I think I may have come up with a solution for it. I'll need to actually implement it and test it, but I'm hoping I can squeeze that in tomorrow, as well.
4. Everything else?
These items are my top priority right now until they are fixed. After that, there's many other general bugs, and then a load of new features and things coming down the pipeline. I'm really excited about those things, but I want to finish this stuff out right and have this running really smoothly for everyone. I really have no idea how many more days of this pre-everything-else work is left, but I will be able to do daily releases like this one with updates, at least.
More to come soon.
Enjoy!
Beta 3.740 Code Panopticon
The idea of a "panopticon" is that it's a structure where there's absolutely no privacy, and there's perfect observation of everything that happens inside. Dystopian thought, right? Well, in this case, I've been working on constructing a panopticon... for memory leaks. The idea is for myself or modders (whoever has caused the memory leak) to be able to get a few core dumps (you can generate those by just hitting F7 now) and then have an idea of what is leaking, which then makes fixing it much easier.
A rather nasty memory leak bug has been lurking for a few weeks now, and I do think I solved it, to be honest... but before you get too excited, there's at least one new one. Or maybe it's a different flavor of the prior one, but the new one acts different, so I suspect it's new. As I've been constructing my panopticon for bugs, there's been a lot of adjustment required in many areas of the code. I clearly have a few blind spots remaining, so it's not a true panopticon yet, anyway.
That said, the game is playable enough for the beta branch, and there's great stuff going on in here, so with a few caveats please have at it! One of the caveats is that if you change around your selection of mods and/or dlcs that are enabled, right now that can cause a lockup or a bunch of errors or work perfectly, variably. It's on my list.
But what else is new? A number of other folks have been super productive, too:
Tom from discord has offered to help with a "bit" of bugsmashing, after examining the open source parts of the code and noting a few things he could fix. He's then been on a real tear throughout the codebase, just fixing tons of items and resolving lots of tickets. That's super helpful, and I'm incredibly grateful!
Among other things, the Fallen Spire are mostly playable again. They're still missing the ability to switch which fleet is being bolstered, and the modular ships aren't in there yet, but Tom has tested and fixed everything else except for the transciever, which is wicked cool and saves me a ton of time I can put into other things for you guys.
Badger decided the AI needs more mean things it can do to you, so he's working on a Svikari variant that can be friends with the AI instead of you. He also made the planet sidebar way more informative when you've got a crippled flagship, versus being misleadingly stuck at one health tick.
Zeus is back in the saddle with a number of balance changes, plus a number of not-listed-yet really cool things for DLC3. One of them involves a gravity well full of orbiting guard posts, which is... just plain awesome.
Daniexpert fixed up the Lost Spire Frigates to "Roam if Instructed" by default, which was a satisfying venture into code for yet another modder/volunteer.
For those with DLC3 early keys, Badger also majorly reworked the Necromancer, and they're ready for some serious testing. Right now it's just the Necromancer helper, so you need to add them as a secondary faction to your main empire and then hit F4 to swap to them. The solo-focused necromancer empire will be coming before too long, and Puffin had some great ideas on that which I'm excited to try out.
I still have a number of code things to fix up in the wake of this panopticon business, but I'm hoping that this will be the last truly large refactor of the game. I'm ready to just focus on bugfixes and more features and such, versus rearranging things in painful detail like this. But things are going to be in much better shape for modders for the long haul now, despite the short term bugs, so I'm definitely pleased about that.
Lots more is coming soon.
Enjoy!
Beta 3.711 More Fixes
I still have to chase the memory leak that we've got going on, and a new rash of reports of various things arrived this evening thanks to Daniexpert (that really is helpful, so thank you!), but this build cleans up some of the issues, and also adds a bit more performance in a couple of areas.
This build also fixes the Civilian Industries mod to once again compile, and I am assuming it works but I have not tested it.
There's still a lot changing at the moment, and will be for another few weeks I imagine, but for now this is kind of "business as usual in the beta." I'm going to take this weekend off, after working the last two.
More to come soon.
Enjoy!
Beta 3.710 Hotfixes Serialization and Performance
Possibly the first of two for today, we'll see.
This one fixes some latent bugs in the prior build, mostly about loading transports and saving the game. You know, not anything important.
This also makes the UI sidebars more responsive, and fixes a possible glitch in xml importing and two definite glitches in necromancer hacking.
The death registry work in general is also moving along towards prep for multiplayer.
Lots of things going on and improving these days, but since we're in this kind of chaotic beta period it's certainly leading to lots of bugs, too. I'll be glad to be out of this beta period when we can be, but at the same time it's still really gratifying how much we're able to improve the core of the game while we're in here. It just makes a terrible mess, as grover would say to elmo.
More to come soon.
Enjoy!
Beta 3.709 Upgrades And Serialization Fix
This one upgrades to a newer version of unity, fixing a number of bugs of relevance, including likely fixing the freeze on startup that was sometimes happening.
Additionally, this release should fix the bug where sometimes units would just turn feral on you when loading a savegame, and all bumrush you for no reason. Apparently testing with the golemite AI faction really makes for a show. If you see this again, please do let us know.
A number of performance improvements have been made in this build, although there's more I wanted to do. I'll continue pushing on with that over time. I also made some adjustments to have a death registry, which has some single player uses (it's related to those performance and correctness bits above), and then also will be the final solution to the ghosts problem in multiplayer once I finish wiring that up.
Final solution? How can that be? Well, the client will be told of things that have died, and the client can ask the host about dead things, as well. That way, the worst thing we can have is not a ghost, or an accidental death -- both two outcomes possible before -- but instead just a "stale data for some reason" issue. That is far less severe, and should be easier to debug if it happens, but even that happening should be way less likely.
SirLimbo continues to be on a roll adding things to the main game and for DLC, working on some changes that affect how mines show their remaining "number of times they can go off," and then also adding a bunch more stuff for the Brutal Guardians in the Dire Guardian Lair for DLC3.
I was hoping to redo some of how transported ships work, or how drones work, while we're in here doing this much surgery anyway. That's been on my wishlist for a long time. Unfortunately... wow. It's just really complicated to try to change it, and it would lead to worse performance if I did. Given I'm trying to make performance go the other direction, I don't feel great about that. It was worth a shot, but I'm going to give it a pass.
I still have some other during-beta things to refactor, to speed up, and to make easier to use in general, but the list is growing shorter. Then it's just a matter of getting the last of the bugs out, making sure multiplayer is going well, and then it's the new-feature train for a while.
More to come soon.
Enjoy!
Beta 3.708 Bugfixes And Death Spawn
This one is smaller, but hits a number of bugs that were bothering people. It also adds some more new features that SirLimbo is working on for some of his new guardians and such.
There is still a serialization bug at the moment (probably), and at any rate I still have more to break in serialization in general. I didn't quite get through my entire list of things I wanted to, but I wound up finding some interesting nests of programming errors and sorting those out. Whether there is a wider issue on that front at the moment or not I am not yet sure, but I have a couple of targets set for review tomorrow.
In the meantime, hopefully things work out well savegame-wise, but please let us know if not. Badger would love some feedback on those new wormhole invasion styles, by the way! Those are in the base game, and are on by default, unlike what I reported last time.
If you're a tester for DLC3, then more feedback and testing with the necromancer and the sappers are also greatly desired at the moment. Lots of changes have happened with them since the last time we got much meaningful feedback.
More to come soon.
Enjoy!
Beta 3.706 Expanded Wormhole Invasions And Raid Engines
(Please note: this one again breaks savegames, and there's a known save corruption glitch from the prior version.)
Well! It's been a busy six days since the last release. This new build mainly has cool new things from Badger and SirLimbo, however, so let's talk about those things first. I tried to fix one bug that could cause a freeze on startup, but other than that I didn't contribute anything here.
First of all, Badger has added options for Wormhole Invasions in the lobby that make it so that you can have something more interesting than the old style of them. Specifically, you can have them just send waves (well, that's a bit different), OR you can have them... you know... drill through the fabric of reality, forge a link between one of your undefended planets and somewhere in deep enemy territory, and spawn several waves at your soft underbelly.
That last option might sound crazy, but it's not meant to be purely punitive. The cool thing about it is that it also changes the nature of the map so that you can now ALSO directly go attack the AI deep in its own territory. Yeah, sure, your underbelly just got exposed, but you also now have a direct link directly into potentially useful territory. If it suits you, you can just reinforce your empire on that side, and use it as a new launching-off point against the AI. Or if it doesn't suit you, you can protect your empire by destroying the projector to shave off that unwanted link.
Remember that you have to turn this option on in the lobby to give it a try. Right now, by default things still work how they did before. If this is popular, maybe this will become a default for some game mode, or maybe it will just stay a niche cool new option; it's hard to say, and we'd love feedback.
SirLimbo, meanwhile, has been expanding the concept of "raid engines" in general by adding a lot of new features that allow for the creation of Dire Guardian Lairs, which was a concept we had in the original game. This new implementation takes it to new levels, though, and the new units that will be involved will be part of DLC3, so I won't say too much about them yet.
For my part, I had two days where I was out of town, and then I've spent all weekend and the first two days of this week completely reworking Arcen's website. That thing was old and creaky, not mobile-friendly much at all, and really not... good at showing the important information. We also had two blogs, which I had started neglecting to update because of how much of a pain it was. I'm still not done with the makeover, but there's a lot of good stuff there, and hopefully it really enhances people's ability to find out what they want to know about Arcen as a company.
I also wrote a couple of blog posts as a response to some questions that some students sent me:
1: https://arcengames.com/what-are-the-different-phases-in-a-development-cycle-when-creating-an-indie-game/
2: https://arcengames.com/what-tools-do-developers-use-to-make-an-indie-game/
3: https://arcengames.com/have-you-noticed-a-change-in-the-way-a-game-is-developed-throughout-the-years/
I'll post a bit more about this tomorrow when I hopefully finish the last of my changes, and then I'll be back to my normal coding. For now, feel free to poke around the revamped site (the game pages have yet to be reworked; if anything seems temp-y or unfinished, it probably is).
More to come soon.
Enjoy!