ColdFusion 9 Reaches End Of Life, Long Live ColdFusion!

NO MORE COLDFUSION 9.0.n SECURITY PATCHES/UPDATES BY ADOBE, AS "CORE SUPPORT" ENDS DECEMBER 31, 2014.

It has been a long known fact that for the ColdFusion 9 series, End of Core Support was coming. Here is the Adobe Support Matrix. According to this article at Adobe's Support Lifecycle Policy this is what Core Support means.

Core Enterprise Maintenance and Support Programs
The existing Platinum Maintenance and Support and legacy Standard Support, Premium Maintenance and Support programs will now provide five years of product support from the general availability date of a product, starting with the release of a ".0" product version (a "root release"). Support for all derivatives -- localized versions, minor upgrades, additional operating systems, etc. -- of a root release terminates with support for the root release. This includes dot and double-dot releases and connector products.
The essence of this is that Adobe will no longer provide updates for the ColdFusion 9 series. Any new bugs or security issues will remain unresolved.

Adobe will still offer "Extended Maintenance and Support" via their Platinum Maintenance and Support services.

Extended Maintenance and Support
This new program option gives your organization an additional two years of Platinum Maintenance and Support services after the five years provided. Extended Maintenance and Support provides your organization the valuable extra time you may need to plan your migration to Adobe's latest technology.

So this is yet another case for upgrading.

Please consider upgrading to a newer version of ColdFusion!

ColdFusion JVM, SSL CA CERTS and POODLE

We ran into this when a company contacted us at CF Webtools with the problem of ColdFusion was suddenly no longer able to connect to their email providers mail servers. The complaint was that ColdFusion was sending emails to their clients just fine the day before but today it can't. These issues are usually best resolved by asking "What changed?". As far as the client knew, nothing had changed.

After doing some investigations on the server we deeded to do some very simple testing. Does it connect to any mail server? Yes, it connected to our mail server without TLS just fine. But it would not with TLS. That's the big clue! I also noted they were on JVM 1.7.0_15 which I know is about two years old. During this time there have been mandated changes to SSL encryption levels to stronger encryption. If the CA Root Certificates on your system are old they may/will not work with new SSL Certs that use the stronger encryption. We upgraded the JVM version 1.7.0_67 and the problems were resolved. ColdFusion was once again send email through their email providers mail server over a secure connection.

So what happened?

[More]

ColdFusion Exploits in the News Again

ColdFusion is taking a bashing in the news this week. I think this is rather typical of the news because they can blame an actual company. Whereas the massive numbers of PHP exploits go unreported in the regular news. There's no company to blame for PHP failures. Just my humble opinion on the motives of the news media.

The truth:
The "New Exploits" are not new, but just newly reported servers that were hit by the same exploit from a year ago. In some cases it was not discovered until recently.

The "ColdFusion BotNet" that is being talked about in the news was taken down last year. The FBI has the person and servers that were collecting data. I have not been able to get any FBI agent to say that exactly, but I have read news reports alluding to the fact. That and the fact that the FBI has been notifying people whose servers have been found listed on the "Botnet control panel". Was there more than one Botnet? Could be?

Most blog posts on this subject are full of hype and lack details. The only blog post that has the details is mine that Mark Kruger, aka. ColdFusion Muse published on his blog. The hack referenced above is two fold. One involving ColdFusion and the other involving IIS.

One of the servers being discussed is currently a client of ours at CF Webtools. They came to us with a compromised server seeking help. We took care of them.

The reminders to make sure your ColdFusion servers are patched just keep coming in! Either patch them yourself, have your hosting provider patch them or if they are not familiar or knowledgeable with ColdFusion contact us at CF Webtools to patch your servers.

UPDATE:
Fellow ColdFusion Guru David C. Epler posted a detailed article on the origins of this exploit in ColdFusion and when it was introduced.

Who Patches Your ColdFusion Servers?

BUSINESS OWNERS: Who is responsible for patching your ColdFusion Servers?

This isn't a trick question. Do you know who is responsible for patching your CF servers?

Do you host your own servers? If so, then you have a Systems Administrator that keeps them patched. Right?

Are you using a Hosting Provider? Are they keeping your servers up to date? Are they keeping ColdFusion patched? Or, is that still the responsibility of your IT team? If you are using a hosting provider, make sure to check your service level agreement (SLA) to see whose job it is to keep your servers patched. In many cases (such as having VPS(s) or Co-located servers) the hosting provider may not do any maintenance. To say that it's better to find this out before you sign an SLA is an understatement. Seriously.

If you're running a company and you don't know the answers to these questions, you owe it to yourself (and your investors) to find out - NOW.

Here's why.
ColdFusion servers are under attack. These are not new attacks. They are ongoing attacks against unpatched ColdFusion servers and it's been going on for well over a year. There are patches for ColdFusion 9 and 10, but versions 8 and previous can be vulnerable, and in fact, are vulnerable. If you're running ColdFusion 8 or older then there is a chance that these patches are not installed according to the Adobe Lockdown Guides (CF9 CF10).

At CF Webtools we've been receiving calls from companies that have had their servers breached; and, while we like the business - $$$ - we'd rather be called before your servers are breached and not after. Patching and securing a server is far easier than trying to repair one that has already been breached. In worst cases, we've had to either replace or reinstall the entire server from scratch.

So back to my opening question... Who is responsible for patching your servers? I'm not trying to scare you or create a panic, but, I am trying to make you AWARE. Servers that get attacked is nothing new – unfortunately, it happens all the time.

But these attacks are something we know about. We know how they're being done, and even more importantly, we know how to prevent them from getting in.

If no one on your IT team is responsible for patching the servers and/or your hosting company isn't patching the servers then, who will? Who will make sure they are secure? We will.

ColdFusion Vulnerability Used to Install IIS Malware

First let me point out that the vulnerability that was found has a patch that has been available since January of 2013. So patch your servers!

While working on servers for our clients at CF Webtools, I found a nasty little Trojan that was slipped onto a client server that was stealing credit card information. I first read back about this type of attack in December 2013 from PCWorld - Attackers exploited ColdFusion vulnerability to install Microsoft IIS malware. The quick rundown is that an unpatched ColdFusion server allowed an attacker to slip a hidden IIS module onto the server and install it in IIS. This IIS Module then steals the credit card data as it passes through IIS. I have the full details posted over on Mark Kruger's, aka. ColdFusion Muse blog.

Yet one more reminder to make sure your ColdFusion servers are patched! Either patch them yourself, have your hosting provider patch them or if they are not familiar or knowledgeable with ColdFusion contact us at CF Webtools to patch your servers.

Mura CMS Vulnerability

Last night Mark Kruger, aka. ColdFusion Muse alerted me to something he found about a Mura CMS take over vulnerability. This is the link that I was sent http://www.securatary.com/vulnerabilities. There's a link to a PDF with exact steps to take over and control any Mura CMS site and server. (Note: At the time I was working on this post there wasn't a fix and the above site wasn't updated with a link pointing to the fix.)

A Quick Summary:
Essentially the attacker knows how Mura CMS works and knows the various URL variables. By appending "?display=editprofile" to the URL they can force Mura to show the user registration form. The attacker needs a proxy that can catch the POST of this form so they can edit the form that is submitted. This can be done with many proxy plugins in FireFox or Google Chrome. Again this attack requires knowing the Mura CMS form values that are passed to create an Admin account.

They alter the form fields like this:

ADD - "s2" value 1
ADD - "type" value 2
ADD - "isPrimary" value 1
CHANGE - "isPublic" value 0

Now they have a Mura CMS System Administrator account. Now all they have to do is login by appending this to the URL "?display=login". From here its up to the attacker as to how much mayhem they want to cause. The mayhem can include uploading a "web shell" which then can allow and and all arbitrary file uploads/downloads.

A Quick and Dirty Fix:
So after figuring out we had Mura CMS sites and the above method would work I took the fastest most direct route to prevent the attack from gaining access to the front door. Using IIS 7+ RequestFiltering I added a DENY Filter for the query string "EditProfile". This prevents anyone from gaining access to the form. I did check the source of the form and it does appear to post to with a "doaction" of "createprofile". It might be a good idea to restrict "createprofile" too in order to prevent a remote form posting or to prevent the attacker using another form via proxy to create a new user profile. As I see it if the form fields can be altered via a proxy and the form action is defined in the form fields then any form post can be hijacked and altered to create a new profile with admin privileges.

If your Mura CMS site regularly allows new users to sign up then this quick fix will break that functionality. If this isn't needed then you should be okay. Otherwise there isn't a fix if a site regularly needs to use the EditProfile feature. Mura would need a fix under the hood that would restrict creating admin accounts to admin users that are logged in.

For IIS versions 5.1 and 6.0 you can use the Microsoft UrlScan Extension for IIS to apply the same RequestFilter on the query string.

With Apache the fastest way may be with "mod_rewite" to detect the query strings and redirect to the home page without any query strings.

Update:
Mura CMS just released a security fix as I was writing this post.

Even after applying the Mura CMS patch you may want to still restrict access to the "editProfile" page via requestFiltering. If there is no need to create new users then there's no need to have that option available. It would be nice if that was a feature you could turn off in Mura.

And I do agree with blueRiver that it would have been really nice if the security researcher had NOTIFIED blueRiver about the flaw BEFORE releasing it to the world. And blueRiver only "wasted" 2 hours getting this fixed from the time they were notified. That's a phenomenal response time! Congrats on that!

ColdFusion 10 on OS X 10.9 Mavericks

Adobe just announced it has fixed the major bug in connector between ColdFusion's TomCat and Apache 2.2.24 and newer.

I just installed the update on my MacBook Pro and the new connector works as advertised! To install the update you need to have the ColdFusion Administrator running (or follow the manual instructions). The internal ColdFusion webserver (running by default on port 8500) has been working so I was able to use the CFAdmin to run the update. After the update in installed you need to run wsconfig as root (administrator) to remove the old connector and then create a new one. *NOTE: If you are doing a fresh installation of ColdFusion 10 you have to run the "Mandatory Update" prior to being able to use the ColdFusion 10 built in update system.

Customer Service, It Mirrors Your Attitude

I use Evernote to type up my blog posts and as a scratch pad for blog ideas. Sometimes ideas get lost deep in Evernote never to be seen again. Today I was digging through to see what notes I had started and to see if I could put together a blog post and I found this little note. It's a funny experience I had a year and a half ago at the doctors office.

From December 2011: The other day I was at the doctors office to have my pre-surgery physical and blood tests completed. The lab tech was an slightly older woman whom was in a particularly foul mood. The new fax machine was causing all sorts of problems and she was expected to make it work without instructions. She's a nurse, not a fax repairman Jim!

In my youth I was a Navy Corpsman and I have a lot of experience doing blood draws. I know this truth to be a fact, you never want the person with the needle to be mad, upset, angry or whatever. Seeing her mood and fearing for the safety of my arm I immediately employed humor and empathy for her frustrations with technology. Not being a fax machine repairman myself I couldn't fix the stupid thing. But I could joke with her about it and share that with her that I work in the IT industry and I can understand the frustrations of these 'darn machines.' After joking with her for a few minutes she began to relax, smile and laugh thus saving my arm from being a Human Stress Relief Pincushion.

When she was done we were both laughing and enjoying life again. Best of all, my arm survived the experience.

A positive attitude goes a long ways and can be a self-defense mechanism.

ColdFusion on Java 1.7

Here at CF Webtools we do a fair amount of hosting and as such I do a large amount of server updates and patching including Windows, Linux and ColdFusion updates. Back in February of 2013 Adobe updated ColdFusion 9 and 10 so those versions will run on Java 1.7 (Java 7). In the following months since then I have not heard much noise or praise from anyone upgrading their servers to Java 1.7. Over the past few weeks I've been making a concerted effort to get our ColdFusion 9 and 10 servers running on Java 1.7. Mainly for security reasons because Java 1.6 has surpassed End of Life as of March 2013.

A few notes up front:
At CF Webtools the ColdFusion 9 servers are all ColdFusion 9.0.1 on Windows servers unless otherwise noted. This article refers to that version and ColdFusion 10. This article by Adobe gives you a matrix of Cumulative Updates for ColdFusion 9 versions. The Java version I am referring to in this article is Java 1.7.0_25 unless otherwise noted.

Why has it taken 6 months to do the update? Mainly compatibility testing. Some of the servers are shared hosting and and there is a lot code to test to make sure it still works. I had to verify that either no java based extensions were being used or that if there was that these worked. And I was hoping to let others be the Guinea pigs. :)

We are able to say that ColdFusion 9.0.1 and ColdFusion 10 are working great on Java 1.7. With ColdFusion 10 I had zero issues at all getting up and running. I installed the new Java version on the server and edited the jvm.config file to point to that version. Note that I had already updated ColdFusion to the latest updates and patches and it's only the Java side that I hadn't updated.

However, with ColdFusion 9.0.1 I had trouble getting CF to start after switching to Java 1.7. Of course I first tried this upgrade on our dev servers so I didn't interrupt our client's businesses. The details of the error when ColdFusion failed to start were lacking completely just "Error loading: C:/Java/jdk1.7.0_25\jre\bin\server\jvm.dll" and nothing else. No additional detail in any ColdFusion logs and not in any Windows logs. This perplexed me. It wasn't until much later and a few cans of cold caffeine that I noticed at the very bottom of the "Cumulative hot fix" instructions this little note:

Note:You could get the following error when starting a ColdFusion instance configured with JDK 1.7:

"MSVCR100.dll is missing."

To resolve this issue, copy msvcr100.dll from {JDK Home}\jre\bin to {ColdFusion-Home}\runtime\bin.

Well I wasn't getting "that" error. I wasn't getting any error details. So on a whim and with high hopes I copied the referenced DLL into the referenced "bin" folder and tried again. This time ColdFusion 9 started. Great! Now for some caveats in Adobes instructions. The resolution they provide is very correct for the "Server Configuration" installation mode. However, for "Enterprise Multiserver Configuration" mode the correct location is "{ColdFusion-Home}\bin." Typically if you went with the installation defaults this is C:\JRUN4\bin. Your server installation location may differ. Once you copy this DLL it will allow ALL your instances on ColdFusion 9.0.1 to start.

Now to add to the confusion. While updating our production servers I decided to wait until I tried starting ColdFusion 9 once just to see if I needed that DLL. All but one server needed me to copy that DLL into place. That has me a tad perplexed. I'm still not sure why all but one needed that DLL copied. This particular server is ColdFusion 9.0.1 Standard. It's not only the Standard edition server that we have so it's not just Standard vs. Enterprise.

I should also mention that we do have a server with ColdFusion 9.0.1 running on Linux and I did not experience any issues upgrading that server to Java 1.7. This DLL issue is Windows only.

Well whatever the cause the good news is we do have our ColdFusion 9 and 10 servers running on Java 1.7 now. So far there have not been any anomalies to report. All I can gather is that if you try to upgrade to Java 1.7 make sure your ColdFusion version is updated to the point that it will run on Java 1.7 and then if it does not start for any reason try copying that DLL into place. That may be all that is needed.

What You Need To Know About CFHTTP, SSL and SNI

While working on a client's website and servers for CF Webtools we ran into a perplexing problem with CFHTTP and SSL. We were working on setting up payment processing using ColdFusion 8.0.1 (yes, I know it's ancient but the client is planning to upgrade to CF10 soonish) and it needed to communicate with a clients .NET server via secure CFHTTP (meaning over SSL). The problem was that SSL communications was failing. The error (below) was I/O Exception: peer not authenticated.

For several hours we tried everything from importing the SSL into the keystore to creating a separate keystore and including it in the jvm.config. We checked name resolution and tried different Java versions. The issue persisted even after upgrading the JVM to 1.6.0_45. We even tested from ColdFusion 9.0.1 and ColdFusion 10u9 running on Java 1.6.0_29 and nothing was working. Usually we can resolve SSL issues in short order. This issue, however, was beginning to seem like something on the server was preventing SSL communications - except for one nagging fact. When using a web browser on the server we could access the payment gateway web service url via SSL with no problem. So SSL was working and all tests indicated that the SSL certificate was installed correctly. What could be the problem?

[More]

More Entries