On my Way to CFSummit 2017

For the first time ever I'm headed to CF Summit. This should be fun and exciting! I'm waiting on Uber to show up and then I'm gone. I just have to get through TSA without a full cavity search. I'll be landing in Vegas around 4pm Vegas time. Let the party begin!

So far I'm thinking these sessions. All subject to change.

9:00 AM - 10:15 AM Day 1 General Session
10:30 AM - 11:30 AM send.Better() - Giving Email a REST
11:45 AM - 12:45 PM Solving problems in ways never before possible, with FusionReactor 7 and FR CLOUD
1:45 PM - 2:45 PM Dockerizing a ColdFusion Enterprise Application, a Case Study
3:00 PM - 4:00 PM Level Up Your Web Apps with Amazon Web Services
4:15 PM - 5:15 PM Power of Simplicity in FW/1 Framework

9:00 AM - 10:00 AM Day 2 General Session: How APIs Accelerate Digital Transformation
10:15 AM - 11:15 AM Application Performance Monitoring Suite in ColdFusion Aether.
11:30 AM - 12:30 PM Securing Mature CFML Codebases
2:45 PM - 3:45 PM Language improvements in ColdFusion Aether
4:00 PM - 5:00 PM CFConfig - A New Way to Manage Your ColdFusion Engine Config
5:15 PM - 5:30 PM Closing Session and Raffle Drawing

Cryptojacking: Hacking for Bitcoins

This is a brief follow up to my previous article on Hacking for Bitcoins in which I detailed how servers were being hijacked with cryptocurrency miners and using your servers CPU power to mine for Bitcoins or other blockchain cryptocurrencies. This is an updated twist on that hack. I saw this Ars Technica article today and it points out that the newer twist is to inject code into your websites code and then process cryptocurrency mining on your website user's computers. This distributes the CPU processing by thousands instead of just taking over a few of your servers.

To do this, hackers are using Coinhive.com which offers an easy-to-use programming interface that lets you setup your own website to process cryptocurrency on your visitors computers. There isn't a requirement to give notice to users that you are going to do this. What hackers are doing is using vulnerabilities in your server(s) and/or website(s) to inject this code in your website. It is estimated that there are about 2,500 websites that are currently compromised and using their users to process cryptocurrency. The fine article at Ars Technica indicates that it appears most are connected to two Coinhive.com accounts. This might mean that the hackers can easily be traced and stopped. But others will surely follow in their path.

How do I know?

When Cryptojacking occurs, a direct side effect is that the website user CPU's are maxed out and system heat starts to increase. This is a tell tale sign that the website you are using is either using your computer for their gain or has been compromised and a hacker is using your computer for their gain. (It could also be one of those annoying Flash based ads that we all hate.) But check the site source code to see if there is anything linking to Coinhive or similar. Ars Technica also reported "Most of the affected sites concealed the connection to Coinhive by adding a link to the domain siteverification.online or one masquerading as a Sucuri firewall."

This is a growing problem and recently Malwarebytes reported that on average it performs about 8 million blocks per day to unauthorized mining pages. People who want to avoid these Cryptojacking scams can use Malwarebytes or another antivirus program that blocks abusive pages

From our point of view at CF Webtools, this is a good reminder to make sure your ColdFusion servers are secure, updated and patched. It's also a good reason as to why your website code (all code really) should be in a secured version control system. That way if something like this did happen to your website code you can replace it from a known clean copy instead of digging through the code looking for the injected code. Additionally, CF Webtools offers PenTesting to check your website code for vulnerabilities. If you need help upgrading your VM or patching your server (or anything else) our operations group is standing by 24/7 - give us a call at 402-408-3733, or send a note to operations at cfwebtools.com.

Hacking for Bitcoins

Are your free CPU cycles making others rich? There's a chance they are and it's at your expense. A recent article at Vice.com states that "At Least 1.65 Million Computers Are Mining Cryptocurrency for Hackers So Far This Year". If this is to be believed then it's possible a server you are running has been compromised and is actually mining cyrptocurrency for the hackers. This type of breach is called Cryptojacking and it's costing you and/or your company money.

Cyrptocurrency is an anonymous, digital currency that is supposed to be untraceable. It's used on the internet to purchase more and more products and services. One of the most common forms of cryptocurrency is Bitcoin. This is from the Wikipedia entry on Bitcoin.

Bitcoin is a worldwide cryptocurrency and digital payment system called the first decentralized digital currency, since the system works without a central repository or single administrator. It was invented by an unknown programmer, or a group of programmers, under the name Satoshi Nakamoto and released as open-source software in 2009. The system is peer-to-peer, and transactions take place between users directly, without an intermediary. These transactions are verified by network nodes and recorded in a public distributed ledger called a blockchain.

Besides being created as a reward for mining, bitcoin can be exchanged for other currencies, products, and services. As of February 2015, over 100,000 merchants and vendors accepted bitcoin as payment. Bitcoin can also be held as an investment. According to research produced by Cambridge University in 2017, there are 2.9 to 5.8 million unique users using a cryptocurrency wallet, most of them using bitcoin. ...

Bitcoin Mining is a record-keeping service that runs on peoples computers, servers, or specialized Mining Devices, that are setup by individuals to help process Bitcoin transactions. As a reward for doing this you are given newly created bitcoins and transaction fees. ie. You can make money by mining for Bitcoin.

This reward is enough that hackers have taken it to the next level and started hacking servers around the world so they can install mining software and use YOUR computers and servers to make money for themselves.

Case Study

CF Webtools has seen this type of hack in the real world. We recently had a company come to us seeking our services for both Server Administration and ColdFusion programming. Part of taking this new company on as a client we performed a security review on all of their servers. They also had existing issues that we needed to look at in particular. Their web server was rebooting multiple times per day at what seemed like random intervals.

Upon review we found the web server was always running at 100% CPU usage with no services claiming to be using that much CPU power. Certainly not ColdFusion or IIS. We decided to install Malware Bytes and scanned for malware. It didn't take long to find that indeed there was malware running on the server. What we found surprised us only because we had not seen this in action before. It was a cryptocurrency miner and it was so intensive that it would crash the server. All attempts to remove the malware failed. It would end up back on the server in a short period of time. The fact is this server was compromised. To resolve the issue we sent one of our decommissioned, but powerful servers, preinstalled with a clean OS to their data center. Then our Operations Manager went on the road to install the new server as well as a physical firewall. We essentially rearchitected their entire server setup. Meanwhile, Malware bytes did it's best to keep the malware at bay while I recreated their web server on the new server. It was a busy week (or more), but we were able to clean the code on the clients server and put that on the new server. We also had to research and rebuild all the web application dependancies from scratch. When it was all said and done we replaced the compromised server with the new one and put all their servers behind a Cisco ASA.

This case of Hacking for Bitcoins proved costly in the end to the company who's systems were compromised all while providing a free profit to the hacker(s).

This is one more friendly reminder to make sure your ColdFusion servers are patched! Either patch them yourself, have your hosting provider patch them. If you need help upgrading your VM or patching your server (or anything else) our operations group is standing by 24/7 - give us a call at 402-408-3733, or send a note to operations at cfwebtools.com.

UPDATE: I saw this Ars Technica article today and it is directly related to this. Cryptojacking is bigger than most people think.

ColdFusion 11 Update 13 and ColdFusion 2016 Update 5

Adobe just released security updates for ColdFusion 11 and ColdFusion 2016. This is a critical security update and you should be updating your ColdFusion servers.

With ColdFusion 11 Update 13 and ColdFusion 2016 Update 5 there are additional manual updates that are required to complete the security patch. The additional requirements are the same for both ColdFusion 11 and ColdFusion 2016 and the remaining information pertains to both versions. Both updates require that ColdFusion be running on Java version 1.8.0_121 or higher. For reference, ColdFusion 11 comes with Java version 1.8.0_25 (* originally it came with Java 1.7.0_nn) and ColdFusion 2016 comes with Java version 1.8.0_72. The Java that needs to be installed is different from the "Windows User" Java client that may already be installed. The installer is available from Oracle. Once the new Java version is installed, the jvm.config file for each ColdFusion instance needs to be updated to point to the new Java version installation path. If you're running the Enterprise version of ColdFusion, there's a likely chance there is more than one ColdFusion instance running.

Part of the instructions from Adobe says that if your ColdFusion server is installed as a J2EE server then there is an additional manual configuration that you ned to do. However, every installation of ColdFusion since the release of ColdFusion 10 is a J2EE or JEE installation. If you do not remember when you installed ColdFusion or you were not the one that did the installation, there are two ways to do the installation; "Server Configuration" and JEE Configuration". What Adobe really means is that if you are using a third party JEE server, "JEE Configuration", and not the built-in Tomcat JEE server, "Server Configuration", then there is an additional step.

If your ColdFusion server is running on a third party JEE server such as WebLogic, WildFly/EAP, custom Apache Tomcat, etc (Not the built in Tomcat that comes with ColdFusion), then the following step needs to be completed.

Set the following JVM flag, "-Djdk.serialFilter=!org.mozilla.** ", in the respective startup file depending on the type of Application Server being used.

For example,

  • On Apache Tomcat Application Server, edit JAVA_OPTS in the 'Catalina.bat/sh' file
  • On WebLogic Application Server, edit JAVA_OPTIONS in the 'startWeblogic.cmd' file
  • On a WildFly/EAP Application Server, edit JAVA_OPTS in the 'standalone.conf' file

This is one more friendly reminder to make sure your ColdFusion servers are patched! Either patch them yourself, have your hosting provider patch them or if they are not familiar or knowledgeable with ColdFusion contact us at CF Webtools to patch your servers.

*Note: ColdFusion11 when it was first released came with a version of Java 1.7.0_nn. Adobe later re-released ColdFusion 11 with Java 1.8.0_25. If you have ColdFusion 11 still running on Java 1.7 I highly recommend that Java be upgraded to Java 1.8. Oracle is no longer supporting Java 1.7 and 1.7 is long past it's end of life. Even though the Adobe instructions for this current security update states that you can run Java 1.7.0_131, I highly recommend upgrading to Java 1.8. Personally I will not install Java 1.7 on a clients servers and sign off on it being 'secure'.

What's In Your CFIDE Folder?

Over the years of working on ColdFusion servers for CF Webtools I have encountered many servers that have been breached (hacked). In most cases the cause was for lack of better description "human error". I say human error because no one properly secured the server when it was installed, no one maintained the server over the years of use and no one was checking to see if anyone had tried hacking the server.

Then something BAD happened that caused EVERYONE to notice the server was breached. Maybe it was your credit card processor informing you that customer cards are being stolen when they purchased from your online store? Maybe it was your servers IP being black listed because it was spewing tens of thousands of spam emails? Maybe you were notified that your medical website had been breached? Maybe it was a notice from the FBI that your server was part of a list of servers that were known to have been breached. Those are NOT good days!

Many times these companies contact CF Webtools for our expertise in resolving breached servers. When they do Mark Kruger, aka. ColdFusion Muse sends me in to investigate, record any data/forensics that I can and mitigate the situation while we simultaneously build a new server for the client. Usually this is what it takes to recover from a breach.

Over time I have collected a large number of 'hack files' that contain the code to breach a website and steal credit cards, entire databases, or even load malware onto an unsuspecting users computer. These files are typically found in an unsecured CFIDE folder. Here are a couple examples.

If you happen to see anything that looks like those files then the server has been breached. If your team is properly securing and maintaining your ColdFusion servers you should never see anything like this. However, if you are seeing files like this in your CFIDE folder or files in your website that are unaccounted for, then it's very likely the server has been breached.

Now what? That is a very open ended question. The first thing to do is accept the fact that it happened and understand that it's happened to companies that are far bigger and with much bigger IT budgets than yours. Remember Target? Now you have to figure out how the breach occurred, determine how much was breached, mitigate the breach as best as you can and then in most cases start building and securing new server(s). It's my belief that once a server has been breached we can never be 100% certain that we've found everything that was put on the server by that breach. Once you have a clean server then you clean and migrate your code and other resources to the new server. This can be a huge and daunting task especially if you have a minimal IT department or none at all.

If no one on your IT team is responsible for maintaining the servers and/or your hosting company isn't maintaining the servers then, who will? Who will make sure they are secure? We will.

Who you Gonna Call? CF Webtools!

Wildcard and SAN SSL with CFHTTP in ColdFusion

Here at CF Webtools are getting a lot of companies coming to us with various CFHTTP issues. Lately this has been happening even more as SSL has been in the news more and certain SSL protocols and encryption levels have gone away or are on the way out. Most recently Wildcard and Subject Alternative Name certificates have been a concern for those running older ColdFusion servers with older versions of Java.

What is a Wildcard SSL certificates versus a Subject Alternative Name (SAN) certificates?

A wildcard SSL certificate allows for unlimited subdomains to be protected with a single certificate. For example if you owned "example.com" a wildcard would allow you to secure www.example.com, mail.example.com, or admin.example.com. Such a certificate would be issued to *.example.com and it could secure any subdomain of example.com for the device on which it was installed.

A SAN cert allows for multiple domain names to be protected with a single certificate. For example the SSL certificate issued to multiple fully qualified domains such as www.domain.com, www.domain2.com, www.domain3.com. This allows for the SAN SSL to be used for multiple sites on the same server all bound to the same IP Address. This does relate back to Server Name Indication (SNI) for web serves that I talked about a while back. In addition, this article by Thawte is a good primer. Let's take them each in turn.

[More]

ColdFusion 8 & 9 CFHTTP still works with Authorize.net

Authorize.net as of May 26th, 2015 upgraded their SSL to SHA-2 (256bit) encryption. See this post. At CF Webtools some of our clients have received this or similar notice.

Important: Security Certificate Upgrades on May 26th

Authorize.Net is upgrading our infrastructure to enhance system performance and security. On May 26, 2015, we are upgrading to new security certificates, which are signed using Security Hash Algorithm 2 (SHA-2) and 2048-bit signatures. Most modern operating systems and web servers support certificates that use SHA-2, however, there is a concern that older software--especially software based on outdated versions of Java--may not.

If any updates are necessary, please refer your web developer to this blog post in our Developer Community, which has all of the certificate information they will need for this update. Our sandbox environment has already been updated so that your developer can validate that your solution will continue to work using SHA-2 signed certificates, prior to May 26th.

After the update is complete on May 26th, any website or payment solution that cannot validate SHA-2 signed certificates will fail to connect to Authorize.Net's servers.

Some have been saying that your ColdFusion 8 CFHTTP using Java 1.6.0_nn will no longer work with Authorize.net. We've found this to not be the case. We have a crack team of ColdFusion Guru's here at CF Webtools and we've been testing for the potential fall out of these and other SSL upgrades for a few months. Personally, I have a testbed system here with ColdFusion versions 8, 9, 10 and 11 and that is setup for testing CFHTTP and SSL issues and I can easily test the same CFHTTP code on each ColdFusion version with each version of Java from 1.6 to 1.8. Another member of our team has been working with support at Authorize.net to ensure the upgrade to their SSL will work with existing ColdFusion 8 on Java 1.6 installations.

[More]

Preventing SSLv3 Fallback in ColdFusion

We've all been taking steps lately to protect our computers and servers from the POODLE flaw in SSLv3. At CF Webtools we've been updating our servers in various hosting facilities to prevent the use of SSLv3. Perhaps you never think about it, but as a ColdFusion developer you make frequent use SSL via various ColdFusion tags or cfscript. For example, CFHTTP lets you access a remote server (such as a web service) with a URL via ColdFusion server and it most often uses SSL in the process.

POODLE and ColdFusion

In case you missed why this is a trending topic (and why security folks like myself and Mark Kruger, aka. ColdFusion Muse are so riled up about it), here is a quick refresher as to what POODLE is according to US-CERT From this article they describe what is affected

"All systems and applications utilizing the Secure Socket Layer (SSL) 3.0 with cipher-block chaining (CBC) mode ciphers may be vulnerable. However, the POODLE (Padding Oracle On Downgraded Legacy Encryption) attack demonstrates this vulnerability using web browsers and web servers, which is one of the most likely exploitation scenarios. "
"This affects most current browsers and websites, but also includes any software that either references a vulnerable SSL/TLS library (e.g. OpenSSL) or implements the SSL/TLS protocol suite itself."

[More]

ColdFusion 9 Reaches End Of Life, Long Live ColdFusion!

NO MORE COLDFUSION 9.0.n SECURITY PATCHES/UPDATES BY ADOBE, AS "CORE SUPPORT" ENDS DECEMBER 31, 2014.

It has been a long known fact that for the ColdFusion 9 series, End of Core Support was coming. Here is the Adobe Support Matrix. According to this article at Adobe's Support Lifecycle Policy this is what Core Support means.

Core Enterprise Maintenance and Support Programs
The existing Platinum Maintenance and Support and legacy Standard Support, Premium Maintenance and Support programs will now provide five years of product support from the general availability date of a product, starting with the release of a ".0" product version (a "root release"). Support for all derivatives -- localized versions, minor upgrades, additional operating systems, etc. -- of a root release terminates with support for the root release. This includes dot and double-dot releases and connector products.
The essence of this is that Adobe will no longer provide updates for the ColdFusion 9 series. Any new bugs or security issues will remain unresolved.

Adobe will still offer "Extended Maintenance and Support" via their Platinum Maintenance and Support services.

Extended Maintenance and Support
This new program option gives your organization an additional two years of Platinum Maintenance and Support services after the five years provided. Extended Maintenance and Support provides your organization the valuable extra time you may need to plan your migration to Adobe's latest technology.

So this is yet another case for upgrading.

Please consider upgrading to a newer version of ColdFusion!

ColdFusion JVM, SSL CA CERTS and POODLE

We ran into this when a company contacted us at CF Webtools with the problem of ColdFusion was suddenly no longer able to connect to their email providers mail servers. The complaint was that ColdFusion was sending emails to their clients just fine the day before but today it can't. These issues are usually best resolved by asking "What changed?". As far as the client knew, nothing had changed.

After doing some investigations on the server we deeded to do some very simple testing. Does it connect to any mail server? Yes, it connected to our mail server without TLS just fine. But it would not with TLS. That's the big clue! I also noted they were on JVM 1.7.0_15 which I know is about two years old. During this time there have been mandated changes to SSL encryption levels to stronger encryption. If the CA Root Certificates on your system are old they may/will not work with new SSL Certs that use the stronger encryption. We upgraded the JVM version 1.7.0_67 and the problems were resolved. ColdFusion was once again send email through their email providers mail server over a secure connection.

So what happened?

[More]

More Entries