Hey all,
As I'm sure you've noticed, the server experienced a lot of turbulence starting on July 18th. It was then turned off on the morning of 19th. The downtime was a result of minor misconfiguration that was done about a year ago and took a while to blow up, making our drives unreliable. That corrupted some of our services, most notably our database server.
During the server's downtime, the last two days I've done my best to restore the most recent version of our database. We obviously back up everything daily, but since I wanted to avoid a large rollback (our most recent full backup was from July 18th 3 AM or so), I managed to manually recover everything up to around July 18th afternoon/evening - the evening before the server went down for maintenance.
To clear some confusion - the downtime wasn't a cause of a DDOS, or any other player attack. The misconfiguration was completely my fault, for which I sincerely apologize.
Despite my best efforts, there's still some minor data loss. This could be virtually any changes (player stats. vehicles, or even accounts) which were created around the afternoon of July 18th or later. If you lost any assets gained in this time and have sufficient proof, you're good to create a refund request. If you lost donation perks purchased in this timeframe, please open a support ticket, regardless of evidence you have. If you lost items that you've had since before July 18th, please create a refund request with at least some sort of evidence and I can try to check it, but if the purhcase happened on 18th I can't do anything.
All in all, with the data corruption that happened, this is a successful recovery. I sympathize with anyone that lost an item or two during the few hours of rollback, but given the situation it could've gone much, much worse if we weren't properly prepared for this - data corruption often leads to complete data loss if it's not backed up properly.
In order to avoid a similar situation in the future, I'll increase the frequency of our backups from once a day to multiple times for the most crucial data (player accounts, vehicles, properties etc) so should something similar happen in the future, the loss will be even smaller. This is the first rollback we've had in the last few years (that I remember, anyway) and I hope it's the last one I'm writing about!
Current donation packages have been extended by 7 days (a week) to compensate for the two day downtime.
Thank you for your understanding, I'll try to make up for the mess
Mmartin
Server's Back up - Minor Rollback
Any important server news or announcements will be posted here.
Moderators: Lead Developers, Senior Lead Admins
Locked
2 posts
• Page 1 of 1
- Mmartin
- Head of Development
- Posts: 7231
- Joined: Thu Mar 01, 2012 10:05 pm
- Ingame name: Lucas Foreman
- Location: Basement
- Mmartin
- Head of Development
- Posts: 7231
- Joined: Thu Mar 01, 2012 10:05 pm
- Ingame name: Lucas Foreman
- Location: Basement
Locked
2 posts
• Page 1 of 1
Jump to
- Server Information
- ↳ NEW FORUMS
- ↳ Server Information
- ↳ Rules & Rule Updates
- ↳ Rules & Rules Updates - Archive
- ↳ Feature documentation
- ↳ Archive
- ↳ Announcements
- ↳ Announcements Archive
- ↳ Server Updates
- ↳ Player Support
- ↳ Official Guides
- ↳ Questions / Technical Support
- ↳ Q&TS Archive
- ↳ Player-Created Guides
- ↳ Survey
- ↳ Surveys Archive
- ↳ 'Happy Birthday' threads
- ↳ I'm new / I'm away / I'm back / I'm leaving
- ↳ New/Away/Back Archive
- LS-RP Teams
- ↳ Forwarded Event Suggestions
- ↳ Archived Event Suggestions
- Links
- ↳ Support Ticket Site
- ↳ Discord
- ↳ User Control Panel
- ↳ SA-MP 0.3.DL
- ↳ faceinvader
- In Character
- ↳ Laws & Acts of San Andreas
- ↳ Laws & Acts of San Andreas Archive
- ↳ San Andreas Network
- ↳ SAN News
- ↳ SAN Podcasts
- ↳ SAN Publishing
- ↳ SAN TV
- ↳ Job Applications
- ↳ Side Job Logs
- ↳ Accepted
- ↳ Archive
- ↳ Classifieds
- ↳ Miscellaneous
- ↳ Events
- ↳ Services
- ↳ General Merchandise
- ↳ Miscellaneous/Misc. Archive
- ↳ Employment
- ↳ Government
- ↳ Employers
- ↳ Jobseekers
- ↳ Employment Archive
- ↳ Real Estate
- ↳ Commercial Real Estate
- ↳ Houses
- ↳ Apartments
- ↳ Complex Websites
- ↳ Real Estate Archive
- ↳ Garages
- ↳ Vehicles
- ↳ Cars
- ↳ Motorbikes
- ↳ Commercial
- ↳ Aircraft
- ↳ Boats
- ↳ Vehicles Archive
- ↳ Personal
- ↳ Personal Archive
- ↳ City Life
- ↳ Businesses / Companies
- ↳ Graffiti/Vandalism
- ↳ Posters
- ↳ Internet
- ↳ Videos
- ↳ Local News
- ↳ City Life Archive
- Out Of Character
- ↳ Faction Report
- ↳ Faction Report - Review
- ↳ Faction Report - Archive
- ↳ Ban Appeals
- ↳ Bug Reports
- ↳ Confirmed Bug Reports
- ↳ Needs Testing
- ↳ UCP Bugs
- ↳ Pending Revision
- ↳ Resolved Reports
- ↳ Trashed Reports
- ↳ Screenshots and videos
- ↳ Random Videos
- ↳ Suggestions
- ↳ Approved Suggestions
- ↳ Pending Revision
- ↳ Implemented Suggestions
- ↳ Denied Suggestions
- ↳ Old Suggestions
- ↳ Board / Forums Suggestions
- ↳ Business Suggestions
- ↳ Faction Suggestions
- ↳ General Suggestions
- ↳ House and House Items Suggestions
- ↳ Job Suggestions
- ↳ Mapping Suggestions
- ↳ UCP Suggestions
- ↳ Vehicle Suggestions
- ↳ Weapons & Drugs Suggestions
- ↳ Forwarded Suggestions
- ↳ Implemented Suggestions
- ↳ Pending Suggestions
- ↳ Trashed/Old Suggestions
- ↳ Community Outreach
- ↳ News
- ↳ LS-RP Awards 2020
- ↳ Website Beta
- ↳ Beta Archive
- Government
- ↳ San Andreas State Government Information & Licensing
- ↳ General Services & Information
- ↳ Government Announcements & Publications
- ↳ SADCR Publications
- ↳ LSFD Press Releases
- ↳ LSGOV Press Releases
- ↳ LSPD Publications
- ↳ LSSD Publications
- ↳ SAGOV Press Releases
- ↳ SADOJ Press Release
- ↳ Recruitment Announcements
- ↳ Police Recruitment
- ↳ Police Department Website
- ↳ PD Recruitment Archive
- ↳ Sheriff Recruitment
- ↳ Department of Corrections and Rehabilitation Recruitment
- ↳ Department of Corrections Website
- ↳ Los Santos Fire Department Recruitment
- ↳ Fire Department Website
- ↳ FD Recruitment Archive
- ↳ San Andreas State Government Recruitment
- ↳ GOV Recruitment Archive
- ↳ Government Contract Center
- ↳ Approved Government Contracts
- ↳ Inactive Contracts
- ↳ Accepted Applications
- ↳ Denied Applications
- ↳ Office of Executive Clemency
- ↳ OEC: Archive
- ↳ State Licensing
- ↳ Bureau of Licensing
- ↳ Business License & Registration
- ↳ Licensed Businesses
- ↳ LIC: Archive
- ↳ Registered Businesses
- ↳ REG: Archive
- ↳ Suspended Businesses & Licenses
- ↳ Archive
- ↳ Aircraft & Landing Registration
- ↳ Registered Aircraft
- ↳ Registered Landing Area
- ↳ Suspended Aircraft & Landing
- ↳ Archive
- ↳ Miscellaneous
- ↳ Protest Permits
- ↳ Approved
- ↳ Denied
- ↳ Hunting License
- ↳ Approved
- ↳ Denied
- ↳ Archive
- ↳ Department of Health: Medical Licensing
- ↳ Accepted
- ↳ Denied
- ↳ San Andreas State Bar
- ↳ Accepted
- ↳ Denied
- ↳ Firearms Licensing Division
- ↳ State Senate
- ↳ Senate Offices
- ↳ Senate Chamber
- ↳ Senate Committees
- ↳ Senate Standing Committee on Commerce, Science, and Transportation
- ↳ Senate Standing Committee on State Codes
- ↳ Senate Standing Committee on Public Safety and State Security
- ↳ Senate Standing Committee on Public Agencies
- ↳ Senate Standing Committee on Appropriations and Budgeting
- ↳ Senate Standing Committee on Judiciary, Rules and Ethics
- ↳ Senate Archive
- ↳ Political Parties
- ↳ State Senate Elections
- ↳ Applications Archive
- ↳ Candidate Campaigns Archive
- ↳ Trash
- ↳ Report A Crime
- ↳ Archive
- ↳ Judiciary of San Andreas
- ↳ Supreme Court
- ↳ Supreme Court Archive
- ↳ Court of Appeals
- ↳ Archive
- ↳ Superior Court
- ↳ Civil
- ↳ Archive
- ↳ Criminal
- ↳ Archive
- ↳ Superior Court Archive
- ↳ Contracts
- ↳ Contracts Archive
- ↳ Publications
- ↳ Publications Archive
- Factions
- ↳ Factions
- ↳ Information, Guidelines & Updates
- ↳ Info/Guidelines/Updates Archive
- ↳ Faction Guides
- ↳ Faction Forum Archive
- ↳ Little Tokyo
- ↳ Filthy Bastards Motorcycle Club
- ↳ S/S VARRIO 38ST
- Businesses
- ↳ Businesses
- ↳ Business Information
- ↳ Archive
- ↳ Company System
- ↳ Billboards
- ↳ Billboards Archive
- Other
- ↳ General Discussions
- ↳ Polls/Surveys
- ↳ Questions and Technical Support
- ↳ Off Topic
- ↳ Forum games
- ↳ Music
- ↳ Dance
- ↳ Metal
- ↳ Rock
- ↳ Rap/Hip Hop
- ↳ Movies / Television
- ↳ Cars
- ↳ Other vehicles
- ↳ Sports
- ↳ Member's Showcase
- ↳ GFX Section
- ↳ Mapping section
- ↳ Scripting Section
- ↳ Stories
- ↳ Members Showcase Archive
- ↳ GTA Modding
- ↳ Official Mods & Modpacks
- ↳ Skins
- ↳ Skin Mod Archive
- ↳ Weapons
- ↳ Vehicles
- ↳ Sounds
- ↳ ENB/Effects/Timecyc
- ↳ Maps/Huds/Crosshairs
- ↳ Environmental
- ↳ House/Clothing items
- ↳ Miscellaneous
- ↳ Guides Section
- ↳ Request a Mod
- ↳ Showroom
- ↳ Showroom Archive
- ↳ Modding Archive
- ↳ Non-English
- ↳ Foreign Moderation
- ↳ Non-English Staff
- ↳ Applications
- ↳ Reserved
- ↳ Archive
- ↳ Archive
- ↳ Archive
- ↳ Albanian
- ↳ Arabic
- ↳ Bulgarian
- ↳ Chinese
- ↳ Danish
- ↳ Dutch
- ↳ Estonian
- ↳ Ex Yugoslavian
- ↳ Filipino
- ↳ Finnish
- ↳ French
- ↳ Georgian
- ↳ German
- ↳ Greek
- ↳ Hebrew
- ↳ Hindi
- ↳ Hungarian
- ↳ Indonesian
- ↳ Italian
- ↳ Korean
- ↳ Latvian
- ↳ Lithuanian
- ↳ Macedonian
- ↳ Maltese
- ↳ Norwegian
- ↳ Polish
- ↳ Portuguese
- ↳ Romanian
- ↳ Russian
- ↳ Spanish
- ↳ Swedish
- ↳ Swedish Empire
- ↳ Arkiv
- ↳ Turkish
- ↳ Ukrainian
- ↳ West Slavic
- ↳ Trash
- ↳ Other Games
- ↳ Action & Shooter Games
- ↳ Battlefield
- ↳ Call of Duty
- ↳ Counter Strike
- ↳ Fortnite
- ↳ Grand Theft Auto
- ↳ GTA V
- ↳ PLAYERUNKNOWN'S BATTLEGROUNDS
- ↳ Rust
- ↳ Rainbow Six Siege
- ↳ Adventure, Roleplay & Strategy Games
- ↳ Dota 2
- ↳ Elder Scrolls
- ↳ Paradox Grand Strategies
- ↳ Runescape
- ↳ World of Warcraft
- ↳ Mobile Games
- ↳ Simulation & Sports Games
- ↳ FIFA
- ↳ NBA
- ↳ Rocket League
- ↳ The Sims
- ↳ Riot Games
- ↳ League of Legends
- ↳ Other Games Archive
- ↳ SA-MP 0.3.DL
- ↳ 0.3.DL Modding
- ↳ Archive
- ↳ Other Games
- ↳ Call of Duty
- ↳ Counter Strike
- ↳ Football Manager
- ↳ GTA IV
- ↳ League of Legends
- ↳ Left 4 Dead
- ↳ Minecraft
- ↳ MMORPG
- ↳ SWAT 4
- ↳ Team Fortress 2
- ↳ World of Los Santos
- ↳ Audio & Audio Packs
- ↳ Los Santos Roleplay UCP
- ↳ Archive
- ↳ -
- ↳ -
- ↳ -
- ↳ -
- ↳ -
- ↳ Amez
- ↳ icrispy
- ↳ Shepherd
- ↳ Samoel
- ↳ About The Shows
- ↳ Chinese
- ↳ Mute
- ↳ Birdman
- ↳ Shammy
- ↳ Accepted
- ↳ Denied
- ↳ Young
- ↳ EMPTY
- ↳ Empty
- ↳ Empty
- ↳ krisk
- ↳ Hungarian
- ↳ Jeff
- ↳ EMPTY
- ↳ Kimberly
- ↳ Los Cycos XV3
- ↳ Ryoma
- ↳ Category 1
- ↳ Mount & Blade
Who is online
Users browsing this forum: No registered users