Everyone talks about “moving everything to the cloud.” In reality, most companies end up with hybrid setups (a mix of on-prem and cloud), and some still keep critical systems entirely on-prem. But why not just put everything in AWS, Azure, or GCP? Here are 5 concrete reasons why 100% cloud isn’t always possible (or smart).
1. Regulatory & Compliance Rules
When it comes to cloud adoption, laws and compliance frameworks are often the biggest blockers. Companies don’t always have the freedom to “ship everything to the cloud and be done with it.” They must comply with data residency, industry regulations, and certification requirements. In other words: where, how, and by whom is the data managed?
Data Residency (Where Data Lives)
First things first, let’s clarify the term “where data lives.” Data isn’t static: it’s collected, stored, processed, and transformed to meet business needs. Every step of this journey must respect specific legal and compliance standards.
- GDPR (General Data Protection Regulation) in Europe requires companies to know exactly where EU residents’ personal data is stored and processed.
- Many governments impose data sovereignty rules, meaning sensitive data must remain within the country, sometimes even in approved or certified data centers.
Example:
A French public administration cannot just move citizen records to a US-based cloud. Under GDPR, such data transfers require strict safeguards, and French law further requires that sensitive records stay in France or with a “cloud de confiance” provider (e.g. SecNumCloud-certified) approved for government use.
Industry-Specific Regulations
Beyond general laws like GDPR, industries also face their own strict compliance frameworks:
- Healthcare (HDS in France, HIPAA in the US): Patient health records must be stored with certified providers. A hospital might keep records on-prem for compliance, but still leverage the cloud for anonymized analytics or AI diagnostics.
- Banking & Finance (EBA/ECB guidelines, PCI-DSS): Core banking systems and payment processing often remain on-prem for audit and security reasons. At the same time, fraud detection or mobile apps may run in the cloud.
- Defense & Aerospace: Classified workloads cannot legally be processed in public clouds. Instead, they run on highly secure, sovereign infrastructures.
- Government & Public Services: Many countries enforce sovereignty rules for citizen data. In France, projects like S3NS (Thales + Google Cloud) were created specifically to deliver a “trusted cloud” compliant with French and European regulations.
Certifications & Standards
Even when companies want to move to the cloud, clients and auditors may demand certifications:
- ISO 27001, SOC 2 → Information security standards.
- PCI-DSS → Payment card industry compliance.
- CNIL (France) → Data privacy watchdog that enforces GDPR and limits how personal data can be exported outside the EU.
Example:
An e-commerce company may run its site entirely on AWS or GCP, but isolate its payment processing system on-prem to simplify PCI-DSS compliance.
2. Legacy Applications & Systems
Another major reason companies don’t go 100% cloud is the presence of legacy applications, old systems built decades ago that are still business-critical today. These applications weren’t designed for the cloud and can’t simply be “lifted and shifted” without serious risks.
What Do We Mean by Legacy?
- Mainframes & AS/400 systems still run in banks, insurance, and government.
- ERP systems (SAP, Oracle) customized heavily over 15+ years.
- Industrial control systems (SCADA, PLCs) in factories and utilities.
- Custom in-house apps written in COBOL, Delphi, or other outdated languages.
These systems work, but they’re fragile, deeply integrated, and costly to change.
Why They Stay On-Prem
- Complexity & Cost of Migration
- Rewriting a 20-year-old core banking system or factory control app isn’t just expensive, it’s risky.
- Companies often can’t afford long downtime or migration failures.
- Vendor Lock-In
- Some software licenses are tied to physical servers or specific hardware.
- Moving them to cloud may break licensing agreements or require huge re-licensing costs.
- Lack of Cloud Readiness
- Legacy apps may not support modern concepts like containerization or microservices.
- Without APIs or modular design, they don’t integrate well with cloud-native services.
Real-World Examples
- Banking: Many large banks still run COBOL mainframe systems for core transaction processing. They’ve tried to migrate, but the risk of breaking mission-critical payments means they keep these systems on-prem, while adding cloud services for fraud detection or customer apps.
- Manufacturing: A factory may rely on SCADA systems tied to physical equipment. These stay on-site, but production data is pushed to cloud for analytics and predictive maintenance.
- Pharma / Healthcare: A pharmaceutical company may run a 15-year-old Oracle ERP on-prem because migrating custom workflows to the cloud version would take years and millions of euros.
- Airlines: Reservation systems often run on legacy mainframes. While mobile booking apps are cloud-based, the “backend brain” remains on old infrastructure.
The Hybrid Approach
Instead of replacing everything, companies often choose coexistence:
- Keep legacy systems on-prem (stable, reliable, but inflexible).
- Extend functionality with cloud-based front-ends or APIs.
- Gradually migrate parts of the workload (data warehousing, reporting, analytics) to cloud.
Example:
A logistics company keeps its old warehouse management system running locally, but builds a cloud-based dashboard on top to give management real-time visibility.
3. Performance & Latency Needs
Not all applications can tolerate the extra milliseconds introduced when traffic travels to a distant cloud data center. For workloads where speed and responsiveness are critical, keeping systems closer to end users or machines is essential.
Why Latency Matters
- Trading platforms: In high-frequency trading, even microseconds can mean the difference between profit and loss. Banks and trading firms often keep these systems in co-location data centers right next to stock exchanges, not in distant public clouds.
- Manufacturing control systems: Robots on a factory line need near-instant responses. If a command goes through the cloud and back, delays could cause errors or safety risks.
- Gaming / VR / AR: Gamers expect smooth, lag-free experiences. Companies deploy edge servers near major cities to deliver faster response times than a centralized cloud region could.
The Edge Solution
- Edge computing brings compute power closer to users or devices.
- Companies may keep local servers on-prem or in regional hubs while syncing aggregated data to cloud for analytics.
Example:
An automotive factory controls robots with local servers for ultra-fast processing, but uploads production data to the cloud at the end of each shift for predictive maintenance analysis.
4. Cost Considerations
Cloud is often sold as “pay only for what you use,” but in practice, cost surprises are common. For some stable workloads, owning infrastructure can actually be more economical.
Hidden & Unexpected Costs
- Egress fees: Downloading data from the cloud (to users, partners, or on-prem systems) can get expensive quickly.
- Storage growth: Cloud storage is cheap at first, but as terabytes accumulate, monthly bills skyrocket.
- Always-on workloads: If a database or VM must run 24/7, the monthly cloud bill can exceed the cost of owning equivalent on-prem hardware.
When On-Prem is Cheaper
- Stable, predictable workloads → like ERP systems that run steadily year-round.
- Massive archives → some organizations find it cheaper to store petabytes on on-prem tape libraries or cold storage than in the cloud.
- Long-term databases → databases that don’t need elasticity are often better left in private data centers.
Example:
A video streaming company uses cloud for global content delivery, but keeps its master video archive in its own data centers to avoid huge ongoing storage + egress fees.
5. Security & Control Requirements
While cloud providers invest heavily in security, some organizations still want direct control over their infrastructure. This can be due to trust, regulation, or strategic reasons.
Why Companies Choose On-Prem for Security
- Data sensitivity: Governments, defense contractors, and R&D labs may not want sensitive data leaving their premises.
- Internal control: Some CIOs and CISOs feel safer knowing their own teams manage firewalls, patching, and monitoring directly.
- Custom security models: Certain industries have very specific compliance or encryption needs not fully covered by standard cloud offerings.
Real-World Examples
- Defense contractors: Classified military data cannot be processed on public cloud. Secure government data centers remain the default.
- Pharmaceutical R&D: Companies developing new drugs may prefer to keep intellectual property on-prem to prevent risks of data leaks.
- Government institutions: Ministries often run hybrid, citizen portals in cloud, but sensitive registries (tax, police, justice) in secured data centers.
Example:
A national defense agency uses cloud for collaboration tools and non-classified services, but keeps its weapons system design data in highly restricted on-prem facilities.
Conclusion
The cloud has transformed the way we build and scale IT systems, but the idea of moving everything to the cloud is more myth than reality. Regulations, legacy systems, latency, costs, and security all create real-world constraints that force organizations to keep at least part of their infrastructure outside public clouds.
That’s why hybrid and multicloud setups have become the norm. Sensitive or latency-critical workloads stay on-premises, while scalable, innovative services run in the cloud. Instead of asking “cloud or no cloud?”, companies are now asking “which workloads belong where?”
The future isn’t about going 100% cloud. It’s about building smart architectures that balance compliance, performance, cost, and security, while still taking full advantage of what the cloud does best.