We are currently performing emergency maintenance on our legacy VPS system: https://vps.nodespace.net. During this time, management of legacy VPS systems will be impacted by slower than usual performance while our maintenance is underway. We're not expecting this to last long.

Update: Maintenance is complete.

We are changing our server manager domain name from manage.nodespace.net to manage.nodespace.com.

During this process, the server manager may be unavailable briefly while the switch is occurring and while we point our systems to the new URL.

This impacts some VPS customers and most dedicated server customers. If you need to perform system administration tasks and the server manager is unavailable, please submit a ticket with your request and we will waive our fee during the maintenance.

Update: This is complete. manage.nodespace.com is the primary domain, manage.nodespace.net will continue to work for the foreseeable future.

Our host node maintenance encountered a kernel panic and we are working to resolve the issue. Please follow the following incident:

https://nodespace.freshstatus.io/incident/346710

Update: This issue is now resolved.

We are currently investigating an issue with our legacy UK2 VPS node.

Update: The issue has been resolved.

We are currently investigating issues with Greywood. There may be intermittent downtime as we troubleshoot.

Issue seems to be resolved.

We are aware of an issue with alfa.boltns.com and we are working to resolve the problem. We will keep you updated with additional details on the incident page: https://nodespace.freshstatus.io/incident/313149 

The issue has been resolved.

We are aware that one of our shared hosting servers is down. We are investigating it.

Maintenance Start: April 29, 2022 at 10:00 PM Eastern Time (UTC-4)
Maintenance End: April 29, 2022 at 11:59 PM Eastern Time (UTC-4)

Maintenance Effects: Lenoir, NC Data Center - Rack 100

Maintenance Details:

We have identified an issue with the PDU in rack LNR.R100. While the PDU is functional, we have lost all remote control of it, including the ability to remote reboot, as well as monitoring. Because of this, we need to replace the entire PDU with a new functional one. During this maintenance window, servers located within in this rack will be offline upwards of about 30 minutes. However, we are planning about 2 hours of downtime in case of any unforeseen circumstances. We apologize about any inconvenience this may cause.

Maintenance Notification: https://status.nodespace.net/incident/171772 

We are currently performing emergency maintenance on one of our rack PDUs in our Lenoir data center. No interruption in service is expected and we're posting this as an advisory.

Incident Link: https://status.nodespace.net/incident/170119 

Maintenance Window: April 21, 2022

Start Time: 09:00 ET
End Time: 16:00 ET
Services Affected: Facility UPS systems
Expected Impact: There should be no client impact.

Details: Please be advised that our facilities team will be performing a UPS maintenance. There is no expected impact during this maintenance. This is an advisory only.

We are patching our servers for the polkit vulnerability. There will be no downtime, no server reboots.

Full maintenance posted here: https://status.nodespace.net/incident/148610 

Our Lenoir, NC facility is currently experiencing an issue with the network. The following incident has been opened for tracking purposes:

https://status.nodespace.net/incident/143377

Update 1: We resolved the first incident prematurely. The network outages have continued on and we have opened a second incident: https://status.nodespace.net/incident/143382 

Update 2: 11:27 AM At this time the network has been stable for about 30 minutes so we feel confident resolving this issue. This issue also brought to our attention that we were hosting some relay infrastructure on our own infrastructure which would prevent customers from being able to access https://status.nodespace.net should our network be unavailable. We have put into place a permanent fix so that the status website is always available, even if our network is down. Apologies on the oversight! We fixed two issues in one.

Server NODE02 was relocated within the data center which caused a small outage. Our DCO team moved the server preemptively. Some services may have been interrupted during this move. We apologize about any inconvenience. 

We're noticing an issue with outbound email. We're currently working on the issue.

Update 1: We are now routing outbound emails through an external SMTP service for the time being. 

We are currently experiencing an outage at our North Carolina data center. 

Update 1: Most services should be restored to shared hosting & VPS customers.

Update 2: We believe that the issue is resolved. Customers still experiencing issues should open a support ticket for further troubleshooting.

We will perform software upgrades and configuration changes to provider edge and core routers in order to facilitate the availability of new services. During this time, you may experience multiple routing reconvergence events or changes in routing. Every effort will be made to limit or eliminate the impact to clients.

Maintenance Window: October 21, 2020
Start Time: 00:01 EDT
End Time: 05:00 EDT

Services Affected:  Network Services
Service Location(s): 801 Main Street NW, Lenoir, North Carolina

Expected Impact: Internet Service interruption of up to 60 mins during the maintenance window

Details:
Please be advised that our network engineering team will be working with one of our carrier providers to provision a new redundant Internet link to add further capacity, diversity and reliability to our Lenoir DC. This change will unavoidably impact service to our customers with data services in our Lenoir NC data center. Our network engineers will ensure that your services will be restored as quickly as possible, however there will be a loss of services during this maintenance which can last up to 60 minutes due to the nature of work required.

Maintenance Window:

Start Time: September 25th 2020 00:01 EDT
End Time: September 25th 2020 06:00 EDT

Services Affected: Internet transit Service

Location(s): Lenoir, NC, USA.

Expected Impact: Re-convergence time of up to 15 mins to redundant paths

Details: One of our carrier providers will be performing extensive work to fix a chronic problem with a long-haul fiber span that has been negatively impacting service to our customers in Lenoir NC. Our network engineers will ensure that the impact to your services will be as minimal as possible, however the reconvergence time will be between 3 minutes to 15 minutes due to the nature of this maintenance.

Maintenance Window 1: Tuesday August 25th, 2020
Start Time: 00:01 EDT
End Time:  08:00 EDT

Maintenance Window 2: Friday August 28th, 2020
Start Time: 00:01 EDT
End Time:  08:00 EDT

 
Services Affected: Network Services
Service Location(s): 801 Main Street NW, Lenoir, North Carolina
Expected Impact: No impact

Details: One of our carriers will be performing required emergency fiber maintenance.  This impacted link is part of our diverse paths, and as such, we expect minimal to no impact. If there is impact, it would be limited to a brief network re-convergence.

Maintenance Window 1: Tuesday August 18, 2020
Start Time: 00:01 EDT
End Time:  06:00 EDT

Maintenance Window 2: Wednesday August 19, 2020
Start Time: 00:01 EDT
End Time:  06:00 EDT

Maintenance Window 3: Thursday August 20, 2020
Start Time: 00:01 EDT
End Time:  06:00 EDT
 

Services Affected: Network Services
Service Location(s): 801 Main Street NW, Lenoir, North Carolina
Expected Impact: Brief re-convergence up to 15 minutes. 

Details: Our carrier partner, Spectrum, will be performing a standard network maintenance during the above listed windows. Our core network has redundant links that will support the changes necessary to minimize downtime and our Network Engineering team will be monitoring our Core throughout the window. Both pre and post checks will be completed to ensure the successful completion of the window.

Maintenance Window 1: Friday August 7th, 2020

Start Time: 00:01 EDT

End Time: 08:00 EDT

Maintenance Window 2: Tuesday August 11th, 2020

Start Time: 00:01 EDT

End Time: 08:00 EDT

Maintenance Window 3: Wednesday August 12th, 2020

Start Time: 00:01 EDT

End Time: 08:00 EDT

Maintenance Window 4: Thursday August 13th, 2020

Start Time: 00:01 EDT

End Time: 08:00 EDT

Services Affected: Network Services

Service Location(s): 801 Main Street NW, Lenoir, North Carolina

Expected Impact: Brief re-convergence up to 15 minutes. 

Details: Our carrier partner, Spectrum, will be performing a standard network maintenance during the above listed windows. Our core network has redundant links that will support the changes necessary to minimize downtime and our Network Engineering team will be monitoring our Core throughout the window. Both pre and post checks will be completed to ensure the successful completion of the window.

Maintenance Window: April 11th, 2020

Start Time: 02:00 EDT
End Time: 05:00 EDT
Services Affected: Internet Transit
Service Location(s): 801 Main Street North West, Lenoir North Carolina
Expected Impact: Up to 15 minutes of re-convergence time to redundant paths

Details: Cogent will be performing network maintenance. During this maintenance window you may experience one or more brief interruptions in service as our Network completes re-convergence.

Services Affected: UPS
Service Location(s): North Carolina Data Center
Expected Impact: No expected Impact
Details: We will be performing maintenance activities on our UPS(s) as part of our normal maintenance cycle. The activity will be split into two nights.

April 6th, 21:00 start time, engineers and our partner vendors will be replacing various components in UPS1 at our Lenoir, NC facility as part of our normal maintenance cycle. It will take approximately 8 hours to complete ending Tuesday, April 7th, 2020 by 05:00 EST.

April 7th, 21:00 start time, engineers and our partner vendors will be performing routine Preventative Maintenance on all UPS’s at our Lenoir, NC facility as part of our normal maintenance cycle. It will take approximately 8 hours to complete ending Wednesday, April 8th, 2020 by 05:00 EST.

There is no power loss expected.

Services Affected: UPS
Service Location(s): North Carolina Data Center
Expected Impact: No expected Impact
Details: We will be performing maintenance activities on our UPS(s) as part of our normal maintenance cycle. The activity will be split into two nights.

April 6th, 21:00 start time, engineers and our partner vendors will be replacing various components in UPS1 at our Lenoir, NC facility as part of our normal maintenance cycle. It will take approximately 8 hours to complete ending Tuesday, April 7th, 2020 by 05:00 EST.

April 7th, 21:00 start time, engineers and our partner vendors will be performing routine Preventative Maintenance on all UPS’s at our Lenoir, NC facility as part of our normal maintenance cycle. It will take approximately 8 hours to complete ending Wednesday, April 8th, 2020 by 05:00 EST.

There is no power loss expected.

We are going to be performing routine maintenance on NODE01.

Recently, a severe bug was discovered in the email software installed on one of our web servers which is hosting your website (on NODE01). In order to resolve this issue, we have patched the software which will only take effect once we reboot the server. We have scheduled emergency maintenance tonight to reboot the server.

Please note: if you have other services with us such as dedicated or virtual servers, these are not affected.

Full details about the bug: https://nvd.nist.gov/vuln/detail/CVE-2019-10149


Thank you for your understanding and we apologize about the inconvenience.

UPDATES:

1. Server has been rebooted.

As part of our commitment to provide the best possible infrastructure for your services, we're sending you this notice to inform you of some upcoming server maintenance in our Orlando Data Center. Please read the following details carefully:

Maintenance type:
Software Upgrade

Procedure:
PHP 5.6 is being officially deprecated as it will no longer receive security updates after January 19th. We will be setting the default PHP version on node01.mco.nodespace.us (the server your website is hosted on) to PHP 7.2.

Latest versions of most web software (such as WordPress, phpBB, MyBB, etc.) already has PHP 7.2 support, however we do recommend testing plugin and addon compatibility. 

If you are running any custom software, please use this time to test for PHP 7.2 compatibility. We've made it easy to switch your site to PHP 7.2 for testing. Please see the following support article on how to enable PHP 7.2 for your site (and how to switch it back): https://my.gonodespace.com/knowledgebase.php?id=174. If everything works as expected, you can reset your site to "inherited". If you encounter errors, we recommend setting the PHP option to version 5.6.

This email is regarding your hosting service with us as we are making a major change in January. As this is a major change, we are sending you this notification as a courtesy so you have time to prepare your website.

NodeSpace does not offer any sort of coding assistance, however our support team is more than happy to assist you with collecting any errors on your site. Please open a ticket through the My NodeSpace portal (https://my.gonodespace.com/submitticket.php) and we'll assist you as much as possible.

Maintenance Window:
Start - 01/19/2019 - 10:00 PM EST
End - 01/19/2019 - 11:59 PM EST

Estimated downtime:
There will be no downtime associated with this maintenance.

Thank you for taking the time to review this maintenance notice. If you do have any questions or concerns about this procedure and how you may be affected, please don't hesitate to contact our support team for more details.

Initial Report

As part of our commitment to provide the best infrastructure for your services, we're sending you this notice to inform you of emergency maintenance taking place in our Orlando based data center facility. We recommend reviewing the following details:

Maintenance type:
Electrical Infrastructure

Reason for Maintenance:
Our monitoring systems alerted our data center technicians to potential issues with electrical systems powering a portion of our Orlando data center facility. In order to proactively address the problems our team of engineers will be running tests to locate any potential flaws and replace any hardware found to be faulty.

Maintenance Window:
Current - ONGOING

Datacenter:
Orlando.DC2

Estimated amount of downtime:
We understand the importance in minimizing downtime while performing any type of maintenance and for that reason we will do everything we can to expedite this process. There are no guarantees, and while we have not encountered any actual power issues affecting your services, it is worth noting that due to the nature of the issue(s) at hand we are considering this an At-Risk maintenance and hence are alerting you to the incident. We do not expect to see any type of downtime but should issues arise power to various sectors of our facility may be affected. We will be sending out a follow up to this notice once the maintenance has been concluded.

Thank you so much for taking the time to review this notice and we do apologize for any inconvenience these unforeseen issues may pose to you and your clients. If you have any questions or concerns about this reboot procedure, please don't hesitate to contact our staff for more details.

Update 1 - 9:51 PM

We thank you very much for your extended patience and simply wanted to provide an update to our prior notice regarding emergency maintenance in our Orlando data center resulting in us placing it into maintenance mode.

At this time our engineers continue to monitor the status of our electrical systems and will continue to do so until our facility is out of maintenance mode. We were able to limit down the factors that triggered our monitoring systems and steps are already being taken to resolve those issues. We remain cautiously optimistic that we will finalize the maintenance with no impact to your services and are aiming to have everything completed by tomorrow afternoon.

As promised, a recap of the situation will be made available after the data center is out of maintenance mode. Lastly, in case an emergency should occur, we do urge you to follow our social media accounts, which are listed below, for updates:

Twitter: https://twitter.com/nodespace

Thank you once again for your patience and understanding during this time and we hope you have a great evening.

Resolution - 11/30 7:49 PM

First and foremost thank you very much for your extended patience and understanding while our team worked on the emergency maintenance over the last day and a half.

At this time we are happy to announce that our team has successfully concluded the maintenance with no reports of downtime and/or power issues and everything is functioning in an optimal state.

In order to be transparent, and as promised, we have included details provided by our engineers as to what prompted the at-risk emergency maintenance below.

At around 11 AM EST on November 29th, our monitors alerted our team to issues with one of our UPS systems as it was failing and eventually did. We were successfully able to transfer the load on the UPS to maintenance bypass and onto full commercial power resulting in the *At-Risk* notice. Upon further investigation the cause of the failure was diagnosed and attributed to a failed capacitor in the system which was successfully replaced. Once that was done we were able to run tests on UPS that proved to be successful and the critical load was transferred back with no issue. After further monitoring everything continues to function as it should and we are concluding our emergency maintenance.

We thank you, once again, for your patience and understanding and assure you that our goal is to always do what we can to ensure you get the services and support that you not only expect, but deserve. As always, if you have any further questions and/or concerns please do not hesitate to let us know. Have a great day!

We have identified an issue with our client portal that is causing some knowledgebase articles to not show up. This issue has been reported to our software vendor for further investigation.

Work Around: In the meantime, please check our Knowledgebase on our support portal site: https://support.gonodespace.com/knowledgebase
Update#1: We have started updating the links to our support site. You can access our support site with your My NodeSpace login.

Resolution: We have resolved the issue and knowledgebase articles are now showing in the customer portal.

We have identified an issue with our Service Desk software that isn't properly importing issues into our customer portal properly. Our software vendor is currently investigating this issue and will supply a patch when it is fixed.

Work Around: To view all important customer announcements, view them within our support portal here: https://support.gonodespace.com/announcements

Update#1: We have started updating the links to our support site. You can access our support site with your My NodeSpace login.

Resolution: We have resolved the issue and announcements now show in the customer portal.

Reason for Outage (RFO)

Location:
NODE01.MCO.DC2

Date:
August 26, 2017

Events:
11:24 AM (GMT-4) During routine server maintenance, the NODE01.MCO server encountered an issue which caused Apache (HTTP) to stop responding to web requests. Our Linux Admins' attempts to bring Apache back online were unsuccessful so they issued a reboot of the server under emergency maintenance procedures.
11:34 AM (GMT-4) The server still had not come back up from reboot. Our Linux Administration Team alerted our NOC team to check the physical server out.
12:34 PM (GMT-4) Our NOC team reported that the server was fine hardware side and provided our Linux Team KVM access.
1:07 PM (GMT-4) Our Linux Admin Team was able to restore services.

Root cause of the issue:
Our Linux Administration team noticed that during server boot up, several of the critical services were not starting. Upon investigation, the root cause was SELinux becoming enabled in the "permissive" state instead of "disabled.

Resolution:
Our Linux Administration team booted the affected server into single user mode and disabled SELinux and rebooted the server normally.

Action plan:
Our Linux Administration team will be closely monitoring the server and reviewing any updates that were automatically installed and working with the upstream software vendors to ensure that this does not happen again.

Total downtime:
Customers on NODE01.MCO experienced 1 hour, 43 minutes of downtime. This downtime is able to be claimed for an SLA credit.

We sincerely apologize about the inconvenience and will do our best to reduce/eliminate future occurrences of the same problems.

Maintenance Type:
Network Infrastructure Maintenance

Action to be taken:
In an effort to maximize the reliability of our services, we will be replacing a line card in one of our core routers at our Orlando.DC2 datacenter.

Maintenance Window:
Start - Wednesday, August 2nd @ 23:00 (11:00 PM EDT)
End - Thursday, August 3rd @ 01:00 (1:00 AM EDT)

Datacenter Location:
Orlando.DC2

Estimated Downtime:
We understand the importance of minimizing the downtime associated with any maintenance. We will take every reasonable precaution to ensure this process is completed safely and without downtime. Our fiber is redundant in our datacenter, so we do not expect any downtime will result from this maintenance. We are notifying you as a precaution. As traffic is re-routed throughout the network, you may experience up to 45 minutes of increased latency, minor packet loss, or possible service interruption during the two-hour maintenance window.

We do apologize for any inconvenience and ask that you please plan accordingly.

Maintenance Type:
Network Maintenance

Action to be taken:
To ensure the reliability of our services, we will be replacing a line card in one of our core routers at our Orlando.DC2 datacenter.

Maintenance Window:
Start - Sunday, July 23rd @ 23:00 Local time (11:00 PM EDT)
End - Monday, July 24th @ 01:00 Local time (1:00 AM EDT)

Datacenter Location:
Orlando.DC2

Estimated Downtime:
We understand the importance of minimizing the downtime associated with any maintenance. Our team will take every reasonable precaution to ensure this process is completed safely and without downtime. Our fiber is redundant in our datacenter, so we do not expect any downtime will result from this maintenance; however, we are notifying you as a precaution. As traffic is re-routed throughout the network, you may experience up to 45 minutes of increased latency, minor packet loss, or possible service interruption during the two-hour maintenance window.

We do apologize for any inconvenience and ask that you please plan accordingly.

Thank you so much for taking the time to review this notice. If you have questions or concerns about this procedure, please don't hesitate to contact us.

Our service desk was temporarily unavailable from the My NodeSpace customer portal. Our technical team found the issue and resolved it promptly. We have manually opened any opened tickets that were misdirected. Any tickets that were opened via Facebook, email, or Twitter were all opned correctly and you should now see these tickets opened within your customer portal. If you have any questions, please don't hesitate to reach out.

We had a brief outage lasting approximately 1 minute during the early morning on July 3, 2017. Our data center identified an issue with one of the core routers providing network connectivity to the data center. The issue has been mitigated and will be resolved during network maintenance.

If you have any questions, please do not hesitate to ask.

Thank you for your patience and understanding!

Scheduled Date and Time: 
Start - December 7th, 2015 @ 8:00PM EST 
End - December 8th, 2015 @ 12:00AM EST 

Maintenance Window: 
4 Hours 

Datacenter Location: 
Orlando.DC2 

Maintenance Type: 
In order to ensure the most secure and reliable services possible, we will be updating firmware on the switch your server is connected to. To complete this process, we will need to power cycle the switch. This will cause a brief period of service unavailability for all servers using this switch. 

Estimated Downtime: 
We understand the importance of minimizing downtime while performing maintenance and for this reason we will do everything we can in hopes that we can expedite the process. As it stands we expect no more than 20 minutes of downtime during the maintenance window as the switch is updated and power cycled. Should there be any complications, we will have personel on-site to address these issues immediately. 

We do apologize for any inconvenience and ask that you please plan accordingly. 

If you have any questions or concerns, please feel free to submit a support ticket.