2131 active members
  219 are Online

Year

25

Day

203

Time

06:35:20

Guest
Login
Sim News RSS Feed Latest NewsArchive
Player Administration

This is why we can't have nice plants
(Posted by Platypus on Year 25 Day 166)
Gorgo Cyclo has been issued with a strike 1 ban for repeatedly stealing items from HerdFest. Year 25 Day 166, 8:28 Platypus banned...
[more]

Ban Hammer gets a workout.
(Posted by Tortle on Year 24 Day 316)
Arnis Endeel has been permabanned for creating yet another account trying to circumvent the ban. Year 24 Day 317, 8:28 Tortle ban...
[more]

Puck Off
(Posted by Platypus on Year 25 Day 139)
Year 25 Day 139, 13:35 Platypus banned player Rick Bender providing the following reason: Golden Rule violation Rick Bender has be...
[more]

Community News

New The Superior Other Sci-Fi
(Posted by Platypus on Year 25 Day 203)
Arise, Sir Shrek, and receive this lightsaber.  Click me for Deathmatch Click me for Grudge Matches
[more]

HerdFest Teleporter Removed
(Posted by Falcon on Year 25 Day 186)
It seems we Platypus forgot to make a Sim News about this... sorry!   The Ithorian Explorer who used to teleport you to the Herds...
[more]

Trying to exist in a post-Herdfest world
(Posted by Mador Anami on Year 25 Day 178)
Hi, I’m Mador Anami. You may remember me from such Quest Team projects as Herdfest, Lightsaber Crafting X and dozens of other entit...
[more]

Technical News

He's No Good To Me Dead
(Posted by Mador Anami on Year 25 Day 201)
Look out master bounty hunters, there's something new in your local bounty shop. Introducing... the Carbonite Freezing Unit. With o...
[more]

Sync Report Y25D197
(Posted by Clarr Solo on Year 25 Day 197)
Expanded Item Claim Items (crated or not) that are inside a character's personal entity or in the wilderness (not a city) may be c...
[more]

Stability Issues with the new Server
(Posted by Clarr Solo on Year 25 Day 197)
I'm making a post to provide some more visibility into the recent server stability issues. Why do we need a new server? We are u...
[more]

Posted by Platypus on Year 25 Day 203 2:20 New

Arise, Sir Shrek, and receive this lightsaber. 


Click me for Deathmatch

Click me for Grudge Matches

 
Posted by Mador Anami on Year 25 Day 201 11:31

Look out master bounty hunters, there's something new in your local bounty shop. Introducing... the Carbonite Freezing Unit. With our state-of-the-art technology, you can freeze your bounty on your ship or vehicle, ensuring they stay perfectly preserved until delivery. No more worrying about escape attempts or unforeseen accidents. Make sure you thaw them out before meeting with your guild rep though. With the Carbonite Freezing Unit, your catch is as good as delivered! Compact, reliable, and easy to use, our Carbonite Freezing Unit is the ultimate solution for every professional bounty hunter.

If you have issues with hibernation sickness, make sure to keep a Medic NPC in your party. They should now be able to help you and your party with any type of medical emergency. If you have any questions, make sure to ask them and they should be able to clarify everything for you.

Thanks to Ondoron Bondoon, Vadik Edik & Thresh Elykson for all their contributions to these new additions.

FAQs:
Q: Can I use the Carbonite Freezing Chamber on all NPCs or players or myself?
A: No, only rescues from bounties at this point.

Q: Why do I have to freeze my bounties now? I liked things the way they were.
A: You do not have to purchase a Carbonite Freezing Unit or freeze your bounties unless you want to. This is merely a fun addititon to free up passenger space that might allow some new ship and vehicle combinations to be viable. 

Q: Do medic NPCs only help with unfreezing?
A: No, I just thought it was a funny segue into the Medic NPCs doing stuff now announcement. 

Q: What do medic NPCs do then?
A: If you talk to a Medic NPC they will explain what needs to happen for them to be able to help. 

 
Posted by Clarr Solo on Year 25 Day 197 20:11

Expanded Item Claim

Items (crated or not) that are inside a character's personal entity or in the wilderness (not a city) may be claimed by that character. This won't apply to crated droids, quest items or admin owned items.

 
Posted by Clarr Solo on Year 25 Day 197 7:04

I'm making a post to provide some more visibility into the recent server stability issues.

Why do we need a new server?

We are upgrading from the old server (ithor, 2015) to a new server (bespin, 2024) primarily to remain on a supported php version and corresponding software stack.

The old server runs PHP7 on apache via nginx, on linux. The old server is hosted directly on AWS EC2.

The new server runs PHP8 on apache via nginx, on docker, on linux. Moving to using docker in the stack allows us to match development processes with production deployment. The new server is hosted on AWS Lightsail. AWS Lightsail is more cost effective for us.

The overall compute performance of the two servers is comparable and does not appear to be an issue, with the new server being significantly faster (when excluding database performance).

Note that currently both www and www2 are hosted on the same physical server, bespin. The difference is only what code is executed.

Database Basics

Both servers connect to the same database, hosted on MariaDB on RDS. The migration to RDS was completed in April 2021.

The database is about 60GB in size, and comprises of about 25 GB of active game data (entities, characters, actions) and 35 GB of archival data (events, transactions). The largest game tables are about 12m rows in size, while the largest archival tables are about 50m rows in size, with a typical retention of about 90 days.

Aside: What is a Vicious Cycle?

In these explanations I used the term vicious cycle to mean any situation where performance degredation causes even more performance degredation. This usually occurs when some task is executed slowly while holding some important resources. Other tasks wait on the resources and also become slow, resulting in everything being slow.

We have some measures in place to try to mitigate vicious cycles;
- Lock Timeouts occur to preemptively kill queries that are unlikely to be successful.
- Lock Timeouts also occur to preemptively kill queries that can cause a vicious resource starvation cycle.
- Deadlocks occur to kill queries that cannot ever be successful due to resource starvation.

Most time that you encounter an issue above, it's because the server has already ran out of resources due to a bug, and its attempting to recover itself without entering a vicious cycle.

Infrastructure Issue #1: Database Latency

The database latency to ithor was approximately 0.4ms, but the latency to bespin is much higher, 1ms, RTT. This has a significant impact to many heavy pages since pages typically do many thousands of tiny queries in succession.

This manifests in large pages (such as the scanner page) feeling slow to load, typically an increase from sub 1 second to 2-3 seconds rendering time of which 98% of the time is spent in RTT latency.

We suspect the increased latency is due to VPC transit between lightsail and RDS. Unfortunately this is not something we can fix by simply upgrading the server size.

Planned Permanent Fix: We will migrate off RDS and find somewhere lower latency to host the database server. We will either host the database server within lightsail database offering, or host the database locally on the same physical server as the webserver.

Infrastructure Issue #2: Database Sizing

Since moving to RDS, the database class was running as an m5.xlarge, however it should have been a smaller class, m5.large. We resized the database class to be m5.large at the start of the new server migrations. The smaller database class has performed more poorly than expected, most likely exaserbated due to self inflicted database issues listed below.

Planned Permanent Fix: Appropriate re-resizing of the server when we migrate the database. Specifically, our database needs a larger buffer cache to hold more of the game state in memory.

SWC Issue #3: Archival Table Bloat

Archival Tables (like events and transactions) have maintenance tasks to trim them to their desired retention length. Typically these maintenance jobs are run every day during daily jobs.

One large table trimming job was disabled during some maintenance work and was never re-enabled causing the table itself to bloat to over 60GB. This severe bloat likely detrimentally affected the performance of the database server.

This issue was identified and fixed by truncating the table (on D197).

Planned Permanent Fix: Partitioning for all Archival tables over a certain size. The archival tables are now large enough that using database partitioning is the only viable permanent solution.

SWC Issue #4: Ineffective Notifier/Events Queries

To generate the flashing notifiers at the top of each page, the events tables are queried using a permission mask, to only find relevant events.

However, there is an issue when a faction member has no privileges in a large faction: the query to find visible events can end up scanning the entire large events table. This slow query can result in a vicious cycle since notifiers are generated by every user on many pages and the resources required to generate notifiers can block other unrelated users from generating notifiers. Notifiers are already heavily cached.

A Workaround was implemented to limit the events scan to 1 Week duration.

Planned Permanent Fix: Partitioning the events tables will reduce the performance impact of regenerating the notifiers. This will reduce the chance that an events lookup query can use too many resources.
Planned Permanent Fix: Decouple notifiers from the database, to avoid notifier failure from cascading between players. This will reduce the chance that when one player fails to generate notifiers, it affects other unrelated players.

SWC Issue #5: Character Creation Table Scans

When a new character was created or certain operations on handles occurred, there were several queries executed that lockup the main player accounts table. Anything that locks a full table has the potential to cause a vicious cycle, specifically since access to the player accounts table is required for generating most pages.

A workaround has been implemented to disable these queries.

Planned Permanent Fix: Player accounts table modification will be decoupled from main page rendering.

SWC Issue #6: PHP Upgrade Backwards Incompatabilities

Due to upgrading between PHP 7 and 8, there are a large number of code incompatabilities that may show up as page crashes or errors. Most visibile of these was issues in vendor code. As of Y25D197 all known compatability issues are fixed or suppressed.

Most PHP8 issues that arose were able to be mitigated by using www2.

SWC Issue #7: Clicking Thundering Herd Issues

A number of pages and buttons (and recent events) allow for a multiplication of required processing power from repeated clicking or refreshing. The most recent server outage today was caused by this.

A few examples:
- Travel buttons can be clicked multiple times causing the same processing to occur multiple times.
- Inventory and WS endpoints can load in parallel.
- Ajax/Javascript functionality can often load multiple times in parallel.

Planned Permanent Fix: UI will be improved to reduce the chance of multiple in flight requests being sent. Currently it is very easy to unintentionally send multiple requests when using the UI, causing server load.

Herdfest Issue #8: Slow Scanners

During herdfest, a large amount of character squads within the same room highlighted issues with the scanner pages. Specifically, rendering some scanner page details required a large amount of server resources.

The scanner was modified to fall back to a simpler version that hid details until manually loaded. This version of the scanner will show up in any room above a certain size.

The quest team moved quickly to redesign the second part of herdfest to reduce the amount of concurrent players per room and this significantly improved server performance.

Possible Future Improvement: Making scanner details loading more streamlined, such as automatically loading them when interacted with.

Herdfest Issue #9: Slow Combat Resolution

During herdfest, a large amount of concurrent combat caused timer resolution to slow. We run combat synchronously since the outcome of combat can make large changes (such as destruction of entities). Combat resolution took approximately 2.2 seconds, however the large amount of players concurrently fighting meant that the tick became saturated at around 3-400 players repeatedly attacking.

We found several potential improvements, but none were deployed during herdfest due to the overall risk of the changes. These improvements have since been deployed with combat resolution currently taking approximately 1.0 seconds.

Conclusion

Thanks for your patience while we resolve these issues.

 
Posted by Clarr Solo on Year 25 Day 191 5:19

www.swcombine.com has been upgraded to PHP 8.3! Which is great! But it might be a bit buggy given it has required over 1,600 individual changes from PHP 7 to get here. It's likely we missed one or two.

If you encounter a bug that you think is related to the php upgrade, please report it on discord, and you can try to workaround the issue by using the other server www2.swcombine.com, which is still on PHP 7.

Expect us to run this setup for about 2 weeks, or until we have ironed out all the major upgrade bugs. After which we will say goodbye to PHP 7 for good.

 

(Also flamethrowers have been nerfed back to their pre herdfest stats).

 
Posted by Clarr Solo on Year 25 Day 188 20:34

Future Warning: Item Claim

Next Weeks Sync will include the following rules change:
Items (crated or not) that are carried by a character or are on terrain (not a city) may be claimed by that character.
This won't apply to crated droids or quest items.


Force Armour & Cybernetics

There is an update for force users and their armor and cybernetics interactions.
The armor you wear and the type of cybernetics you have will negatively impact the force alpha level and maximum Force pool.
The equations for reductions are on the rules page.


Combat changes

- Only leaders can initiate the attack, not party members (or non-leader squadron members).
- Leaders no longer need to have weapons of their own to initiate attack, as long as the squadron has at least one weapon.
- You can attack squads with non-combat enabled ships in them, the non-combat enabled ships will be ignored. (Example: a Frigate squadded with a fighter, the frigate will be ignored). Previously this would have been rejected with an error.

 
Posted by Falcon on Year 25 Day 186 18:34

It seems we Platypus forgot to make a Sim News about this... sorry!

 

The Ithorian Explorer who used to teleport you to the Herdship has gone home. The rewards vendor is still available and can be found in the cockpit of the Herdship which you can travel to at the following coordinates:

Sector: Bright Jewel
System: Bright Jewel (-5, 185)
System Position: (12, 9)

We have no plans to remove the rewards vendor any time soon, so if you still have leaves burning a hole in your pocket, feel free to spend them!

Platypus Edit: Sorry, my bad! One note - the Herd Meet Host in the Bafforr Jubilee's entrance can still send you home one last time. This NPC will be removed after D194.

 
(Edited by Platypus on Year 25 Day 187)
Posted by Clarr Solo on Year 25 Day 179 0:43

On Y25 D183 there will be server maintenance. We will be retiring the main server server known as ithor and replacing it with a new server named bespin. We thank ithor for its service, o7.

Edit: Delayed till Y25 D183.

Edit: Maintenance has begun. You should not see many, if any, changes from the darkness/player side of things. If you have an error that will not go away, please notify us via #swc-help or via support ticket

Edit: Maintenance has finished. Server has been upgraded.

 
(Edited by Clarr Solo on Year 25 Day 183)
Posted by Clarr Solo on Year 25 Day 181 20:36

Force Choke

A new dark side power has been added: Force Choke.

Bugs:
- Many changes removing obsolete code related to non-modular factions.
- Fix exploit with garrison attacking.
- Medical UI fixes for starting medical actions while duplicate actions are occuring.
- More Journal Fixes
- Missing Creature rules infobox
- Removal of Monthly Jobs: Alazhi Regen and Purchasable NPC restock have been moved to weekly jobs. Jobs have been scaled.
- Fix for city decay: No longer blocked by cities with only wrecks.
- Fix issue with large image link not displaying.

 

FALCON EDIT: I've run the Alazhi and NPC population jobs an additional 3 times so they produce their expected monthly output, but from here on out it will be weekly. 

 
(Edited by Falcon on Year 25 Day 182)
Posted by Falcon on Year 25 Day 178 11:01

Because I've been asked this many times recently:

 

SQUADDING A COMBAT ENABLED SHIP TO A NON-COMBAT-ENABLED SHIP TO PREVENT COMBAT IS A GOLDEN RULE VIOLATION. 

 

If you are doing this currently, please fix yourselves. I would hate for your ships to mysteriously explode for no reason...

 

 

edit: docking is ok

edit edit: Same rules of Derra, Serroco and Kowak apply to Arvala when it comes to trapping players.

 
(Edited by Falcon on Year 25 Day 179)
Posted by Mador Anami on Year 25 Day 178 18:29

Hi, I’m Mador Anami. You may remember me from such Quest Team projects as Herdfest, Lightsaber Crafting X and dozens of other entity scripts.

I hope everyone had a good time at Herdfest. Most of the teammates who worked on Herdfest had not been involved in any previous events before. We are already painfully aware of a few issues to improve on for future events. But we also wanted to get feedback from all of you.

Here’s the feedback form: https://docs.google.com/forms/d/e/1FAIpQLSdfRTLZnEAbxJ_Gjro7iVhGe5xAaaPqfXpp3HVpwvEh642yJw/viewform?usp=sf_link

I have also had a number of people reach out to me asking about joining the quest team. I appreciate all the enthusiasm. The main thing I’m looking for is for you to show me a LISP script that you wrote which shows you understand the basic fundamentals of the language and have some fun and creative ideas. If you aren’t familiar with LISP, luckily you should have lots of rare protocol droids from Herdfest you can test things on.

Rilan Ludel has made an awesome intro to LISP video tutorial series. https://www.youtube.com/playlist?list=PL1Ega_eBm2fH3XFQXEBqDGlAP82kqrTde

Kay Dallben has a written tutorial overview as well. https://docs.google.com/document/d/11i9y8ydhDKJmIAr6Ah-g-sRJ4hEqCKeYg0V2ZM4sdEA/edit?pli=1

Even if you think you would excel more in the quest writing/story side, having a basic understanding of LISP will help you to know what’s possible.

Feel free to reach out to me via DM or on Discord if you have any questions.

 
Posted by Kolomon Seph on Year 25 Day 174 17:27

Greetings sentients!

Just a quick update today in honor of Herdfest ending...

Ithorian Herdship - New large and small images

Enjoy!

Kolo