020 3095 0005
Service Announcements
This page holds information about upgrades, enhancements, and service problems which impact either positively or negatively and significant group of oRe Net customers. Customers will be notified individually about significant issues with their own service.
Current Information
- 03/05/2025: Probable networking outage affecting our primary colocation.
- 21/02/2023: Switch outage with our upstream provider affecting our primary colocation, multiple services will be down until it is resolved.
- 16-20/04/2022: Software updates on private cloud hypervisors impacting virtual machine services briefly, out of hours
- 06/04/2021: Installation of an automated power switcher into our primary data centre to provide a degree of redundancy to power feeds for resilience. Very brief periods of inaccessibility resulted.
- 21/11/2020: Major upgrade to primary mail server, short outage expected. Significant amounts of prior testing has been carried out, and ongoing monitoring will be close after completion.
- 18/11/2020: Precautionary network maintenance upstream had unexpected consequences causing problems reaching our core services from certain networks. Resolved in fairly short order with reconfiguration both sides.
- 23/02/2020: Data center move suffering ongoing problems, services will be restored from backups as soon as possible, likely today.
- 22/02/2020: Data center move in the afternoon affecting colocated and private cloud services, including access to send/receive email for a few hours. We are doubling the capacity of our private cloud in the process, all being well, so performance improvements expected.
- 01/04/2019: Work to increase available disk space on primary mail server briefly went awry, service quickly restored, lessons learned.
- 06/03/2019: SSL certificate changed, do not be alarmed by messsages about it. Some automation issues causing intermittent problems briefly. Now resolved.
- 06/02/2019 onwards: Migration of websites and tertiary DNS/MX to a new server. No significant interruption expected.
- 04/02/2019: Power glitch caused private cloud cluster problems.
- 23/06/2018: Power maintenance causing a substantial outage.
- 06/06/2018: Re-attempting phase 1 of hypervisor memory upgrade.
- 13/05/2018: Phase 1 of a planned hypervisor memory upgrade, no significant customer-facing impact expected. Maintenance aborted.
- 18/04/2017: Outage of our primary data centre for a period during the evening, rendering email, colocation, and some other services unreachable.
- 25/06/2016: Major power issue at our primary data centre will have rendered email and some colocation services inaccessible this morning.
- 28-30/05/2016: Due to errors made by a previous supplier (which we have now worked around and fixed) some users will have suffered email deliverability issues during this period.
- 28/05/2016: Data centre move, of critical infrastructure, mail and other services will be degraded (some colo services and webmail will be down) for up to a few hours this afternoon.
- 08/03/2016: Upgrade to primary mail server spam scanning systems to hopefully stop the flood of spams people have been experiencing. Testing has been done but customer impact remains possible, select users have been asked to monitor.
- 05/11/2015: Primary mail server software updates, no negative ongoing customer impact expected.
- 02/11/2015: Increasing disk space on a couple of service instances to avoid future errors.
- 01/11/2015: Filesystem problems on the primary mail server caused some mail problems.
- 05/08/2015: Filesystem errors on the webmail and misc web hosting machine caused an outage, resolved by rolling back to a recent backup.
- 07/06/2015: Primary mail server moving to our new private cloud infrastructure.
- 31/05/2015: Webmail and various charitable things will be down for maintenance during today.
- 05/05/2015: Routing issues ongoing. Everyone is infuriated. We are being let down badly by multiple vendors here.
- 04/05/2015: Routing issues persist for the new network setup for the below migration, resulting in some people having trouble accessing servers. The matter is in hand and should be resolved today.
- 03/05/2015: Migration of numerous servers to a new location, colocation and virtual hosting services will have been variously effected in the afternoon. There may be residual effects on DNS for a while, but they should clear up soon.
- 26/05/2014: Migration of primary web server to a new hosting location may cause minor inconsistencies in website data (none expected though).
- 11/02/2014: NTP amplification attacks against one of our London facilities (again), caused notable degradation of service to that facility for prolonged periods. Now resolved.
- 18/06/2013: Two outages of primary mail server, delays to email will have resulted. New hardware order initiated to replace this system with a faster and more reliable one.
- 23/05/2013: Power failure in one of our racks resulted in the primary web server being down for a brief period early morning. Some virtual hosting customers may have experienced a loss of web service briefly.
- 02/05/2013: Internal failure within one of the mail nodes. Some mail may have been delayed.
- 29/03/2013: Hardware failures and other issues with the primary web server resulting in a prolonged period of downtime while the majority of the hardware was replaced, now back up.
- 27/09/2012: Transit outage at one of our facilities effecting some websites briefly.
- 29/03/2012: Upgrading the primary web server to fix instability issues took longer than expected. Customers will have experienced some downtime today.
- 10/03/2012: Primary mail server has failed and hardware needs replacing, this may take some time.
- 12/01/2012: Some form of major internet issue in the UK affecting various services. Primary rack hosting webmail was down for around 30 minutes. All now resolved.
- 21/08/2011: Residual issues have been resolved.
- 20/08/2011: Some residual issues with email, customers are recommended to us the webmail system in the short term if they encounter problems.
- 19/08/2011: Software update of the primary mail server to improve brute force detection resulting in some downtime.
- 05/04/2011: Software update of the webmail systems for better syncronisation with PDA/phones.
- 17/02/2011: A software update of the primary web server is planned this evening. If this proceeds, there will be a short period of downtime out of hours of this server.
- 17/02/2011: Brief period of downtime for one of the servers to perform a hardware upgrade. The only significant customer impact would be a brief loss of webmail access.
- 24/01/2011: Customers may be experiencing delays in receiving email due to primary mail server load brought on by excessive amounts of incoming bounced messages, spam, and resolver issues at that site having knock-on effects on email.
- 23/12/2010: Load levels on one of our servers have become unacceptably high, and a lot of work is being done to get this sorted. Customers may experience intermittently unavailable webmail.
- 13/12/2010: Upgrades to all servers, colocated and shared in response to recent serious security advisories.
- 05-06/11/2010: Migration of oldest non-RAID web server to a RAID architecture, which overran resulting in some noticeable downtime of some customer websites, now resolved.
- 31/10/2010: We have been experiencing ongoing issues rebuilding the RAID array detailed on 27/10/2010 causing some downtime, we are working on resolving this as quickly as possible with minimum of interruption, and will likely be abandoning this method of RAID provision in the future.
- 27/10/2010: Replacement of a failed disk in the RAID array of one of our servers will cause downtime for the webmail system (and other facilities for which there are backups, thus not impacting customers) for a while out of hours this evening.
- 09-22/09/2010: Continued tweaks to the work done on 08/09/2010, no significant customer impact expected.
- 08/09/2010: Further enhance anti-spam measures on the primary mail server, no significant customer impact expected.
- 03/09/2010: Implement additional anti-spam measures on the primary mail server, no significant customer impact expected.
- 21/08/2010: Refresh of SSL certificates for webmail/mailhost/smtpmail, and upgrade of webmail system software.
- 21/07/2010: Power outage which cause major issues for the UK internet as a whole may have affected availability of some services around midday, or early afternoon.
- 02/06/2010: The outage was resolved by 09:30, so impact on the working day should be minimal.
- 02/06/2010: Major london outage of the upstream transit for one of our racks, affecting primary mail server, and numerous colocated servers, rendering them inaccessible. Support tickets have been opened and the issue should be resolved promptly.
- 30/11/2009: Planning to upgrade the secondary ('freebie') web server this evening.
- 08/11/2009: Old transit connection to the THM facility turned off, no significant customer impact expected, as all required reconfiguration has already been done.
- 14/09/2009: Brief mail outage due to authentication system issue recurring during routine maintenance.
- 20/08/2009: SSL certificate regeneration on the primary mail server. Customers may have to click yes to changed certificates, but otherwise no significant impact.
- 13/08/2009: Supplier changes have necessitated the IP address of the primary mail server being changed. You may need to adjust your firewalling.
- 12/05/2009: Add more rulesets to the antispam system in order to catch more junk. We will be monitoring the situation to make sure that they are behaving correctly.
- 14/04/2009: Changed the antivirus system and other upgrades to the primary mail server. There was a brief period in which SMTP service from it was unavailable.
- 18/12/2008: Upgrades to the primary web server. No ongoing negative customer impact expected.
- 14/12/2008: The maintenance from 13/12/2008 triggered further problems with the primary mail server, which is now back online. No mail has been lost, although customers may find some of their 'message has been read' flagging is lost. No other ongoing problems expected, the server has been substantially upgraded, so better performance is expected though.
- 13/12/2008: Signficant maintenance to the power supply at our Telehouse Metro facility, which will mean reboots of all equipment there, and some downtime, notably of webmail and other email services. We are also hoping to substantially upgrade the primary mail server in the same window.
- 26/11/2008: Apologies to any customers attempting to reach us by phone in recent days; our beloved incumbent telecoms operator (BT) saw fit to change the main telephone number for oRe Net without actually telling us. We are now back to the regular number.
- 07/11/2008: Network failure affecting all telehouse metro colocation and managed server hosting clients. Our upstream provider seems to have had a problem which would cause reachability issues overnight/early-morning from some of the UK.
- 20/04/2008: Upgrade to mail scanning systems of the primary mail server. We hit a strange bug, and have contacted the vendor regarding a fix. Customers may have observed intermittent access to mail services and some delay in email coming through. Now resolved.
- 03/04/2008: Brief period of downtime of the primary web server for fan erplacement and checking, to mitigate the likelyhood of a future unpleasant failure.`.
- 11/03/2008: Upgrades to tertiary mail servers in order to prevent 'backscatter' junk mail, no significant customer impact expected.
- 08/03/2008: A period of downtime of the primary web server due to supplier issues with virtually no notice, now resolved.
Planned Work
- A program of security enhancements to all webservers is still under way.
Prior/Resolved Notices