At St. Joseph’s Health in New Jersey, a Drive to Optimize IT Infrastructure

The four-facility St. Joseph’s Health is based in Paterson, New Jersey, and has been serving patients in northeastern New Jersey since 1867. The health system encompasses St. Joseph’s University Medical Center, a quaternary state-designated trauma center and state-designated St. Joseph’s Children’s Hospital (651 adult and pediatric beds) on the Paterson campus; St. Joseph’s Wayne Medical Center (229 beds), an acute care community hospital on the Wayne campus; St. Joseph's Healthcare and Rehab Center (151 beds), Cedar Grove; Visiting Health Services of NJ, Totowa; and more than 30 North Jersey community-based facilities.

Jesse Fasolo, director, technology and information security, at St. Joseph’s Health, is responsible for developing and implementing the technology strategy to align with the health system’s strategic goals, as well as the evaluation of technical resources to ensure technical operation, while also overseeing the information security program.

Fasolo and his colleagues at St. Joseph’s have been partnering with the Mountain View, California-based Pure Storage, to optimize their organization’s IT infrastructure. Recently, Healthcare Innovation Editor-in-Chief Mark Hagland interviewed Fasolo regarding St. Joseph’s current strategic IT trajectory. Below are excerpts from that interview.

How many are on your staff at St. Joseph’s, and what is your reporting structure?

I have 130 staff members working with me, and I report directly to our CIO and work closely with her.

Tell me about your organization’s current IT landscape.

I came to St. Joseph’s with nearly 20 years’ experience in multiple industries, including legal, retail, finance, and healthcare, and along the way, I’ve built a good understanding of data architecture, network architectures, and security. And nearly five years ago here at St. Joseph’s, we initially identified that all the core IT infrastructure was under-invested or lacking in current or future capabilities or requirements.

In addition, there was a full leadership change in the organizational change that brought about a new organizational strategy and vision. So when I arrived, we developed a new technology roadmap, looking at technology, inclusive of storage, of networking, of all the components that make up the infrastructure. So aligning the technology roadmap to support the goals of the organization, is how we started out. We decided to deliver on all new infrastructure to support the business, but without downtime, without spending a huge amount of money, and without spending a huge amount of money all at once.

So I was able to identify equipment, vendors, and really build out my plan, really focusing on holistic, combined infrastructure. So we delivered a reference infrastructure with all the components, and it’s infrastructure that can stand the test of time and be founded on layers of redundancy.

Far too often, I see people implementing a solution that then has to be changed again as soon as a new technology is available, and it’s a waste of time and money. Specifically with Pure, I knew in advance, and the public releases of what’s coming down the pipeline have been crucial, and the communications with the folks at Pure. So with them, I was able to build an architecture around some of the specific releases that Pure was planning, including the Active Cluster technology, and Cloud Snap.

Can you explain how you’re incorporating those solutions?

Yes, the Purity Active Cluster is an architecture that allows workload to be running on either side, at any time. In the past, you’d have to buy all the equipment and infrastructure, servers, etc., all at the same time, as hyper-converged systems. So Pure offered an Active Cluster architecture that supported workloads across two environments at the same time. So there was a significant cost differential involved.

What does that look like?

As far as our current deployment with Purity Active Cluster and VMWare MetroCluster, we’re able to seamlessly fail over automatically, with no human interaction, to achieve a full-site recovery from Site A to B. And if something happens on one site—we have two data centers—it’s running automatically at the other site. That’s the gist of the Purity Active Cluster. And we’re in a full virtual environment here. Almost all, if not all, of our technology, is set in this environment.

And in terms of data protection, among the requirements we articulated were RPO and RTO—recovery point objective and recovery time objective, for our information systems. In the past, our recovery-time objectives involved days. With this active-active environment, [the tolerance of end-users for downtime] has really become zero. Now, if there were to be a catastrophic failure, everything’s already running. So there’s a very big advantage in having the environment running simultaneously in two environments; that’s been very helpful. And Pure is also fully encrypted, whereas our legacy solution had not been encrypted. And then along with the Pure environment comes the ability to do snapshots seamlessly—a journal for backup and recovery. Snapshots are a vital part of ensuring recovery.

How often are you backing up your systems, and how often auditing backups?

We backup every day. We do snapshots on the array on both sides, every 15 minutes, a full snap, and then we do four a day, and we keep them for weeks, as points to recover. And within the first week of migrating to this environment, we had to do a recovery of a particular server, and while that was happening, we were able to bring this snapshot online and go back live within minutes, restore an environment back to four days ago. And a snapshot is a version of the system in time, where I can refer back to it. It does the entire pod or environment, where I can reference, almost like a catalog: I can go back 15 minutes ago, or a day, or a week ago, and go back to that version.

How often do you audit your backups?

We have operational tasks to audit and review backup successes or failures on a daily basis; we have automated reports. And in terms of backup schedules, we do full backups every day.

Do you do behavioral monitoring?

We do. We have a bunch of products that scan the environment for user-level changes or Internet egress activity. We use multiple platforms constantly scanning both our server environment as well as our data. We use Digital Guardian, Crowdstrike, and Palo Alto Firewalls, as well. And then our Pure has the ability to look at all these snapshots and recognize these changes. We’re very conscious of ransomware and malware, so part of the product suite is offering the ability to offer those changes; we’re looking to upgrade and make use of that in the short term.

Another area for us is agility. We talked a bit about automatic recovery. And really, staff does not need to follow any run books with the Pure environment; it’s fully automatic, which means we can sleep at night. We can enact a full data center to data center failover as an exercise.

And within one week, we had to mount a snapshot and recover a system. In previous legacy storage environments we migrated from, in those environments, all systems would be impacted; if backups took place, the entire system would slow down. In this new environment, there’s zero impact on end-users. In the past, they would complain of slowness when the environment was hit by storage work. And this gives us the ability to do full disaster recovery planning and testing. And we’ve migrated into a more mature stance, from a risk and compliance perspective.

What have been the biggest lessons learned in all this so far, for you and your colleagues?

The biggest lesson learned on the journey so far, overall, is that in healthcare, in this environment, technology is always in demand, and it’s truly the foundation, and enables the clinical environment to move forward. Simply stated, there’s no building that can be supported without a good foundation, and over the past three to five years, I’ve helped us implement a foundation to support the clinical operations and enterprise. In healthcare, you have to be able to deliver new solutions, and faster than ever before. And now we have the capabilities of delivering on performance and on system-ready environments. And everything is becoming artificial intelligence-equipped or machine learning-equipped, and we have the full capabilities of providing that.

How do you see the differences between healthcare on the one hand, and the retail and financial services sectors, which you had previously worked in, on the other hand?

The areas of focus I oversee include AV services, telecommunications, data center, which covers server, networks; I also oversee the operators. In the other industries I come from, most of those technology stacks are similar. But the use is different. In finance, they have the capital to acquire advanced systems; if you needed a product, the money was available, and you simply went out and acquired the product. In the non-profit HC environment, it’s very difficult to source the capital, as well as to communicate the value of the investment. In HC, some of the technology investment is not seen as a service that can be delivered to the patient. So a lot of the technology is foundational; is a core component that people don’t realize provides the opportunity for everything else to run on. I see it shifting a bit, where having a robust network and solutions that can deliver unimaginable performance compared to in the past, is starting to help people value what I do, and what we do. It’s been a long journey. And as part of that, we implemented a whole HER system, and I had to establish wireless capability across the enterprise, and had to increase network bandwidth to 10 or 40 gig capability at all the sites. Knowing when and how to line those up, along with an EHR [electronic health record] implementation, was challenging. And imagining what the environment will be like 10 or 15 years from now, in healthcare, is ever-changing.

Is there anything you’d like to add?

There is a physical-plant aspect to this. The vendor whose solution we replaced had had six full racks of equipment implemented, and we consolidated that down to three racks of space when we made the transition; that’s a huge consolidation. That set us up to migrate our data to other environments easily. And over the last year, we’ve implemented a third data location. We’ll have three locations, to be able to seamlessly flip data center to data center and do a full migration. And we’ve realized improved performance. Computer log-in times for nurses to doctors have been reduced from a minute and a half, to ten to twenty seconds to open an app. And every minute counts in healthcare; and that helps the overburdened clinicians.