Imagine a claims team at a general contractor preparing a delay claim.
The evidence sits in daily logs, RFIs, schedule updates, subcontractor quotes and two years of email. A frontier model would probably reason through that material better than anything else available. It is also the last place the commercial lead wants those files to go.
That tension is common.
When construction practitioners were asked to rank the challenges of adopting generative AI on projects, data privacy, security and legal liability came first, ahead of poor training data and the lack of construction-specific knowledge [1].
A 2026 systematic review of 142 studies on large language models in construction reached a similar point: projects hold drawings, BIM models, contracts and operational records, and relying on cloud-hosted models raises the risk of leakage and unauthorized access [2].
Outside construction, Cisco’s 2024 survey of 2,600 privacy and security professionals found that 27% of organizations had banned generative AI at least temporarily, while 48% admitted that non-public company information had been entered into these tools anyway [3].
So the practical question for a project organization is rarely “frontier or local?” It is closer to “which part of this work needs the frontier, and which part should stay home?”
The gap is real, but it is measured in months
Frontier models are better, and the race at the top is hard to follow.
Still, Epoch AI’s capability index shows the best open-weight models, the ones you can download and run yourself, trailing the best closed models by about four months on average since January 2026 [4].
That gap matters most for the hardest reasoning. Many project tasks are different: reading a specification section, extracting dates from correspondence, classifying defects, drafting a first summary. For those, a model a few months behind the frontier may well be good enough, and a team can test that on its own documents.
Construction researchers have already started to split the work this way. Jeon and Lee built a defect question-answering system in which a closed cloud model generated the training data, while a fine-tuned open-source model handled the sensitive defect records. Their hybrid version scored highest of the approaches they compared [5].
Singh and Hsieh used commercial models to generate and augment construction schedules, then used small, locally deployed models to evaluate the results, so sensitive data did not need to leave a secure environment [6].
How a hybrid workflow handles a sensitive question
Here is the idea applied to the claims example. The steps below are an illustrative workflow, not a tested product.
An illustrative hybrid workflow. The sensitive reading happens locally; the frontier model sees only a cleaned question.
The files stay on a machine the organization controls. A local model reads them, finds the relevant events and builds a timeline.
Before anything leaves, the local model replaces sensitive specifics with placeholders. Party names become “Owner” and “Sub A”, dollar values become ranges, and project identifiers disappear.
Only the cleaned question goes to the frontier model, for example: “Given this sequence of notices and delays under a typical stipulated-price contract, what are the strongest and weakest points of an excusable-delay argument?”
The answer comes back in placeholders. The local model maps it to the real records and flags any point it cannot trace to a source document.
A person reviews and approves the result.
Each model does what it does best. The frontier model contributes reasoning about structure; the local model does the reading, which is where the sensitive volume lives. Research on this local-remote pattern supports the economics too. In the Minions study, a small on-device model read long financial, medical and scientific documents while a frontier cloud model broke the task into smaller subtasks. The best protocol recovered 97.9% of the frontier model’s accuracy at about 18% of its cloud cost [7]. That study measured cost and quality, not privacy or construction work, but it shows the frontier model does not need to read everything to contribute.
Two qualifications matter. Redaction is imperfect: a timeline with names removed can still identify a project to someone who knows it, so decide what is too revealing even in placeholder form. And local is not automatically secure. A desk machine still needs access control, encryption and updates. The benefit is that your organization decides where the data goes, not that the risk disappears.
A private machine is also a laboratory
The second argument for local AI is less about secrecy and more about learning.
Adopting AI in project work takes a lot of trial and error on real data. Which instruction extracts change-order dates reliably? Does a model read your company’s daily-log format correctly? How often does it misread a specification reference? Those questions are best answered on your own past projects, which are exactly the files people hesitate to upload.
A machine on the desk lowers the cost of asking. A team can run the same task a hundred times on a closed project, compare models and keep every output for review, with no per-token bill and no approval needed for each upload. Mistakes made while testing stay inside the building.
What a desk-sized AI computer actually buys
Aside from custom AI rigs, two machines currently define the high end of local AI hardware, and they make different bets. Three numbers explain the difference:
Memory decides how large a model fits. At 4-bit precision, a rough rule is half a byte per parameter, so a 400-billion-parameter model needs about 200 GB before any working space.
Memory bandwidth decides how fast the answer appears. Generating each word means reading the model’s active weights, so more bandwidth gives faster output.
Compute decides how fast long inputs are read, how quickly a model can be fine-tuned and how many people can share the machine.
Apple’s new Mac Studio with the M5 Ultra goes after the first two. It offers up to 512 GB of unified memory with 1.2 TB/s of bandwidth, Apple claims up to 4.3 times the peak AI compute of the previous M3 Ultra, and several machines can be clustered over Thunderbolt 5 [8]. The M5 Ultra starts at US$5,499; the 512 GB configuration went on sale this month at $22,299 [8,9].
NVIDIA’s DGX Spark goes after the third. Its GB10 chip is rated at up to 1 petaFLOP of 4-bit AI compute (a theoretical figure that assumes sparsity), paired with 128 GB of memory at 273 GB/s [10]. It runs the same CUDA software stack used in NVIDIA’s data centers, which matters if a prototype will later move to larger infrastructure. Memory shortages have pushed its price from $3,999 at launch to $6,950, and NVIDIA added a 64 GB version at $4,999 this month [11].
The benchmarks follow the bandwidth. Tom’s Hardware measured the M5 Ultra generating text almost four times faster than the DGX Spark on a dense 27-billion-parameter model, with faster prompt processing as well [12]. LMSYS’s review of the Spark shows the other side of the trade: a 70-billion-parameter model processed its prompt at 803 tokens per second but generated only 2.7 tokens per second, which the reviewers judged better suited to prototyping than production [13]. The Spark’s total throughput rises sharply when many requests run at once, and it can fully fine-tune an 8-billion-parameter model on the desk [10].
So the competition is between capacity and generation speed on one side, and compute and ecosystem continuity on the other. A reasonable reading for project teams: a large-memory Mac suits a few people running big open models interactively; a Spark suits teams that want to fine-tune, serve many small requests, or build something they later deploy on NVIDIA servers. Neither replaces the frontier. Both make the local half of a hybrid workflow practical.
A small experiment to start
You do not need the most expensive machine to begin. Pick one closed project and one repeatable task, such as extracting delay events from daily logs. Run it with a local open model on the original files, and with a frontier model on a redacted version. Record accuracy against an answer a person prepared, the time spent correcting each output, and anything the redaction failed to hide. That single comparison will tell you more about where your organization’s line between local and cloud belongs than any benchmark.
Explain It to a Friend
What if every question your team asked AI were visible to your competitors? People would ask fewer and safer questions, and the organization would learn less.
What a team is willing to show limits what it can learn.
Local AI moves that limit: it gives a team a private room where clumsy questions, failed prompts and half-formed ideas cost nothing. If privacy is what makes honest trial and error possible, then experimentation is the natural way into this new domain, not something to wait on until the tools mature. So cheers to experimentation, done somewhere safe enough to be wrong. How would you explain this to a friend?
Notes
This post is part of the PM Along AI series. See all episodes here.
Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry’s future.
Subscribe to EPM Research and learn how to receive future PM Along AI posts.
Support us by sharing this article with your friends and colleagues, or over social media.
References
[1] Fernández, A. V., & Shen, Y. (2026). Prioritising generative AI adoption challenges in construction projects: A Fuzzy Analytic Hierarchy Process approach. International Journal of Construction Management.
[2] Gao, Y., Yiu, T. W., Shen, X., & Tam, V. W. Y. (2026). Large language models in smart construction: A systematic review of implementation strategies, applications and future directions. Engineering, Construction and Architectural Management, 33(15), 159–181.
[3] Cisco. (2024, January 25). More than 1 in 4 organizations banned use of GenAI over privacy and data security risks: New Cisco study. Cisco Newsroom.
[4] Edwards, J., & Emberson, L. (2026, May 29). Open models lag state-of-the-art closed models by 4 months. Epoch AI.
[5] Jeon, K., & Lee, G. (2025). Hybrid large language model approach for prompt and sensitive defect management: A comparative analysis of hybrid, non-hybrid, and GraphRAG approaches. Advanced Engineering Informatics, 64, 103076.
[6] Singh, A. K., & Hsieh, S.-H. (2026). Multi-LLM-based augmentation and synthetic data generation of construction schedules and task descriptions with SLM-as-a-judge assessment. Advanced Engineering Informatics, 69, 103825.
[7] Narayan, A., Biderman, D., Eyuboglu, S., May, A., Linderman, S., Zou, J., & Ré, C. (2025). Cost-efficient collaboration between on-device and cloud language models. Proceedings of the 42nd International Conference on Machine Learning, PMLR 267, 45682–45719.
[8] Apple. (2026, August 25). Apple introduces new Mac Studio with M5 Max and M5 Ultra. Apple Newsroom.
[9] VideoCardz. (2026, October). Apple Mac Studio with 512GB RAM now costs $22,299, memory upgrade alone is $15,500. VideoCardz.com.
[10] Jain, D. (2025, October 14). NVIDIA DGX Spark review: The AI appliance bringing datacenter capabilities to desktops. StorageReview.
[11] The Register. (2026, October). Nvidia debuts $4,999 DGX Spark with half the RAM and storage, amid memory crunch. The Register.
[12] Freedman, A. E. (2026, September 21). Apple Mac Studio (M5 Ultra) review: Local model citizen outpaces DGX Spark and Threadripper. Tom’s Hardware.
[13] LMSYS Org. (2025, October 13). NVIDIA DGX Spark in-depth review: A new standard for local AI inference. LMSYS Blog.
The part that stood out to me is that the frontier model only ever sees a cleaned question, while the local model does the reading, where the sensitive volume lives. That's the right division of labor, and your two caveats (placeholders can still identify a project, and local isn't automatically secure) are what make it credible.
I'd push the framing one step further. Once open models are only a few months behind and good enough for the reading, extraction and first drafts, the local machine stops being the cautious fallback and becomes the default, with the frontier as the specialist you consult. As I put it in a recent piece, "Raw model inference is one part of that system, not the whole of it." The node, its memory, its tools and the person reviewing the result do the rest, and you reach for the cloud by choice, not by necessity.
https://www.pond.is/p/the-gap-closed-why-open-weights-make