As many of you are aware, we started enforcing a new policy for password rotation.  In theory this was a splendid idea from both a security and administrative view point.  Over the past 24 hours we have had a wide range of customer comments regarding this new policy.  These have ranged from this is the best new policy ever all the way to TotalChoice has gone to the dogs.

The largest concern about this new policy came from our reseller client base.  This is due to the fact that the new password change can not be effected from WHM.  It can only be effected from cPanel logins.  This is a flaw or bug that we did not realize was present in cPanel until our reseller clients started running into it.  We are working with cPanel in an effort to correct this issue.

As many of you are aware I take customer service very serious and I also do not beat around the bush with my clients.   It appears that this policy has become more of a nuisance for clients and that is something I would never want to promote.  It is with that in mind that I have chosen to revert this change and forgo this new policy.

I would like to thank all the clients that have reached out in both support of and against this issue.

As always thank you all for making TCH what it is today and most importantly….

Happy Hosting!

I would like to thank all the clients that have reached out in both support of and against this issue. Please review the latest update concerning this policy.

Dear TCH Family,

TCH takes security very seriously and we are always looking at ways to improve the security for our internal infrastructure and for our client’s web presence. In an ongoing effort to provide our clients with the most secure environment possible we are enforcing the following new password rotation rules.

Effective immediately, every shared and reseller hosting client upon their next login to cPanel will be prompted to change/update their user passwords.  You will not be able to utilize your old password during this new password update.  Once this is completed, users will then be prompted every 180 days to once again rotate their passwords.

We would suggest you implement at a minimum, an 8 character long password with combinations of upper/lowercase, number and special characters.

We believe this new policy will insure that the hosting environment provided to our clients will become more secure.  This will also promote password security that will hopefully pass down to the way all users handle their passwords not only here at TCH but overall.

As always we appreciate your business and look forward to providing our clients many more years of happy hosting!

Thank you

Bill Kish
TotalChoice Hosting

This does not effect any servers across the TCH network.  This only effects the main totalchoicehosting.com pages.

This issue cropped up at 2:38PM EST.

We are working as quickly as possible to restore services in full.

Update 3:56PM EST: All databases have been restored from backups. The website and back end services (billing, forums, helpdesk) are all back online.

Enough of my rambling, everyone that hosts at TCH knows about our budget hosting with excellent up time and services.  Even with the two outages of last week our network up time is still 99.982%.  In fact, TotalChoice is ranked #1 by Hyperspin for hosts with more than 50 servers monitored.  Number #1 uptime of all monitored hosting companies.  This is quite a feat for a budget hosting company.  I am really proud of TCH and its staff.

Whilst our up time is excellent , we have learned something from our short periods of down time.  We need to move our network status page and blog to another Data Center.  Currently if the TCH Data Center goes off line, so do our status pages.  This is going to change.  We are in the process of preparing to do just that.  I have a friend that owns and operates another data center on the West Coast.  He was looking to do the same and wanted to host his status pages off network.  So we have struck a deal.  A server for server type of barter.

We expect to roll out these changes in the next 48-72 hours.  I will be sending out a mass email to our client base informing them of this after we have done a bit of testing on it to make sure everything is working as it should.

Happy Hosting!



The first thing we need to understand is what is being attacked; as the post subject implies, it is primarily Joomla being attacked as the software has had a series of 9 vulnerabilities released since the 1st of September of which a number of more in depth attacks have formed around. The intended purpose of most of these attacks is to taint web sites with injected javascript, that code takes advantage of a number of client side browser vulnerabilities that if not patched or stopped by an antivirus can cause further issues for web site visitors.

Now, at a glance you might be thinking that if someone fails to patch web site software then it is there own problem, how does this affect me? That is where the dl() function comes into play, the dl() function is essentially a dynamic loader for php modules or 3rd party extensions. To simplify this a bit, the dl() function when enabled allows anyone to add extensible features onto php, generally these are all well to do features but if someone so desires they can create a dynamic loader module with malicious intent.

The scenario we are looking at is that attackers have gained entry to vulnerable web sites, primarily through joomla then they upload a series of malicious scripts including a dynamic loadable module for php that once enabled through dl() has the ability to inject javascript code into pages. The code usually finds itself placed before the body tags and executes its payload on a visitors first visit to a site, a cookie is then set that expires every 2hours then the payload executes itself again on a new visit.

This attack though had far reaching implications, only affected 4 servers on our network (denver, dantooine, alderaan, chewbacca) of which only about half the sites on the given servers or in some cases less were being tainted by the attack. As alarming as this situation is, we need to stress that no content was actually modified on sites except the joomla sites themselves that were compromised.

The way we have come to deal with this situation is a layered approach, we have first and foremost made increased efforts to identify compromised sites on our servers and suspend/remove them. The next step was to cut off the enabling function of the attack, which is the dl() function. This function was actually something we used to disable on servers for its malicious implications but over time that procedure was phased out in the interest of allowing users to install custom dynamic loadable modules from their home directories such as ioncube. However, now that ioncube is standard server-wide on all servers, there is little in the way of other commonly installed packages that depend on dl(), php.net has even went as far as to declare dl() deprecated as of php 5.3.

With dl() disabled on servers, the effects were immediate and all reports of tainted sites stopped, now when I say stopped I do not just mean that that lightly. We literally sat around all evening bashing the f5 key on our keyboards trying to get the javacode injections to reappear on sites, between myself, Bill and Dick we must have done over 6 hours of combined keyboard kungfu in this effort. It was with great relief that we were not seeing anymore reports or issues ourselves first hand but it was still not quite enough to actually be confident that we had done enough.

We are continuing to be extra vigilant with compromise assessment on the servers to prevent any further malicious content from being injected into sites, in addition to this we have on some servers started to use suPHP as a basis for new php security standards. Essentially, by using suPHP we enforce php code to run as the user who executed it instead of as the web server but it goes beyond that by enforcing strict permissions on content and not allowing anything to run above mode 755 (such as world writable data) and also making sure that executed content is owned by the user. This might seem problematic however since the code is now executing as the user, there is no longer a need for data to be set to mode 777 (world writable) or its ownership set as the web server user, which reduces support issues and vastly increases security. The suPHP changes are something we have only rolled out to about 6 servers so far but the support issues it has generated are minimal for the advantages it provides, in the future we will be looking to roll this change out to more servers on a slow but steady basis.

That is where we are at, if you have any questions or concerns regarding this blog or the topics discussed please feel free to comment or head to the TCH forums for further dialog.

Update: Authorize.net is now back online. You will be able to make payments. We will update you if this or any other changes arise.

As a Team Mom/Mother of three little leaguers and an avid Astros’ fan, I know this gnawing urge for a WIN all to well. I have seen the hurt of defeat in a child’s eyes and the smile of victory on their faces. I’ve watched as they have picked themselves up out of the dust just as the Umpire shouts out “Safe”. I’ve followed the ball as it flew into outfield and fell perfectly into the Left fielders glove, when they were only inches away from second base. They have learned to overcome the losses and bask in the glory of the game that is Baseball.

When they are not on the field themselves, we are perched in the stands of  Minute Maid Park, the home of Houston Astros’. We are a loyal Astros’ family, and attend as many games as we can. Each year we spend Mother’s and Father’s day with these guys. Today, April 6, 2009, is Opening Day of the 2009 Series. The Astros’ will go head to head with The Chicago Cubs.

Roy Oswalt, who has been with the Astros’ since 2001, will be pitching the opening game. This year’s 40 man roster will consist of new and old faces of the organization. Lance Berkman and Hunter Pence, my favorites, will be with us once again and are currently part of the active roster. We are all looking forward to another exciting season.

So whether it be a Little League field or Minute Maid Park, as they say “Let me root, root, root for the home team”.

PLAY BALL!!!

I personally think it is a conspiracy between my wife and my doctor to torture me, although they claimed to only be concerned with my health. Despite my paranoia,  I decided they had a legitimate point and sadly drank my last Dew a few weeks ago.

Well it only took a couple a of days before the crankiness set in and the throbbing pain of what my doctor so lovingly called “caffeine withdrawal headaches”. Caffeine headaches? Really? The paranoid side of me started to wonder what they really put in Mountain Dew? I shook off that crazy thought and eventually came to my senses, put away my tinfoil hat, and begin to look for an alternative fix.

As I frantically rummaged through the kitchen, I came across an old dusty coffee pot hidden in the back of the pantry.  Eureka, a source of caffeine with no sugar, I can’t possible go wrong here!  I never have been a big coffee drinker but that was about to change!  Since we only break out the coffee pot for company, it took a bit more rummaging before I was able to find what I needed.  One stale can of coffee-it will have to do for now, paper filters, and one coffee cup from the “good dishes” I am not allowed to use. I never understood why we needed a set of “good dishes” just for company, but I digress.

Intent on curing my headache with as much caffeine as possible, I quickly begin the task of setting up the coffee pot. Within a few minutes the kitchen was filled with the smell of fresh brewed coffee. Stale coffee or not, the aroma was overwhelmingly intoxicating, which only intensified my craving. While it only took a few minutes to brew-it felt like an eternity- with every drip of the coffee pot slowly torturing me. I wonder what Attorney General Eric Holder would have to say about that?

The coffee pot then began to gurgle as if it was dying, signaling me that I could finally have a hot cup of caffeine coffee. It wasn’t the greatest cup of coffee, it was very bitter tasting but it was serving its purpose and it didn’t take long for the effects of the caffeine to kick in. As started on my second cup,  I could feel my heart rate increase and the throbbing in my head begin to fade away. I had soon emptied the entire pot(8 cups)and happily found the way to my office.

I am sure all that coffee was not good for me, but I was feeling so much better that it didn’t really matter. It also made for a very productive day-I think-as I bounced off the office walls from my caffeine high. I was now talking nonstop as I was fielding the phone calls that seemed to come in one after another. I do apologize to the brave souls that called me that day, you must have thought I was crazy and I  hope you can forgive me for rambling on. I can’t say that I didn’t enjoy this day and the energetic feeling the coffee provided, but I knew it was going to wear off soon-resulting in that typical caffeine crash.

This all occurred about 6 weeks ago and I am still starting my day with a pot of coffee and  just a bit more self control.  It doesn’t seem very wise to replace one vice with another, so I have been considering cutting back on the coffee too.  Although, I doubt that will happen any time soon.  Some would say I just have an addictive nature but I prefer to think that if you enjoy something-then enjoy it as often as you can.

I need a refill…

One of the great benefits of hosting with TotalChoice Hosting, is being a true part of our family.  I can not express how many times a confused client has approached me inquiring about an account upgrade.  Just recently I have been helping a client with a large amount of shared hosting sites make some choices on how to upgrade.

Mike Ford, owner of newyorkmustangs.com , has been a happy TCH client for over 5 years.  During this time Mike has grown from one shared hosting account to over ten shared accounts.  Each time adding a new account with a smile. Mike was ready to order his 11th account, however this time, he approached TCH asking about his options.

TotalChoice offers many upgrade options.  I explained to Mike that moving into a reseller account would be the next logical step.  This would allow Mike to host unlimited domains within the allotted disk and bandwidth space.  I explained that a reseller account was still on a shared server. However, Mike really wanted to have full control.  He wanted to add as many domains to his account as humanly possible.  I explained to Mike, that a fully managed dedicated server was the way to go.

After a few days of back and forth questions, Mike looked around and shopped the competitors.  In the end, Mike choose to host his new dedicated server at TotalChoice Hosting.

Thank you Mike! I know you will be pleased with your new server!

Check out Mike Ford’s website at

http://www.newyorkmustangs.com

Happy Hosting!