<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[EPM Research: PM Along AI]]></title><description><![CDATA[PM Along AI is an EPM Research publication dedicated to practical AI prompting, workflows, tools, and developments for construction and project management students and professionals.

Our purpose is not simply to automate today’s work. It is to help students, educators, researchers, and industry professionals learn, work, and grow alongside AI.]]></description><link>https://epmresearch.com/s/pm-along-ai</link><image><url>https://substackcdn.com/image/fetch/$s_!tkur!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F86a44a6a-597c-498e-a9d8-a107e101727a_549x549.png</url><title>EPM Research: PM Along AI</title><link>https://epmresearch.com/s/pm-along-ai</link></image><generator>Substack</generator><lastBuildDate>Tue, 17 Nov 2026 00:54:32 GMT</lastBuildDate><atom:link href="https://epmresearch.com/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Pouya Zangeneh]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[epmresearch@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[epmresearch@substack.com]]></itunes:email><itunes:name><![CDATA[Pouya Zangeneh]]></itunes:name></itunes:owner><itunes:author><![CDATA[Pouya Zangeneh]]></itunes:author><googleplay:owner><![CDATA[epmresearch@substack.com]]></googleplay:owner><googleplay:email><![CDATA[epmresearch@substack.com]]></googleplay:email><googleplay:author><![CDATA[Pouya Zangeneh]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[009 - How to Design AI Systems for Project Management with Learning in Mind]]></title><description><![CDATA[Feedback, memory, and evolving procedures can help today&#8217;s AI systems carry useful lessons into their next decisions.]]></description><link>https://epmresearch.com/p/009-how-to-design-ai-systems-for</link><guid isPermaLink="false">https://epmresearch.com/p/009-how-to-design-ai-systems-for</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Mon, 09 Nov 2026 17:04:25 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!7urL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On one project team, &#8220;approved&#8221; means the people and budget are ready. On another, it means funding is available and staffing comes later. An AI assistant working with both teams could collect plenty of accurate information and still give poor advice. Its usefulness depends on carrying each lesson into the right situation.</p><p>Thinking in terms of learning makes us specify what should change, where the lesson belongs, and how we will check whether it helped.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!7urL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!7urL!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png 424w, https://substackcdn.com/image/fetch/$s_!7urL!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png 848w, https://substackcdn.com/image/fetch/$s_!7urL!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png 1272w, https://substackcdn.com/image/fetch/$s_!7urL!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!7urL!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png" width="936" height="468" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:468,&quot;width&quot;:936,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;New evidence redirects a decision while retaining a visible record of its earlier path.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="New evidence redirects a decision while retaining a visible record of its earlier path." title="New evidence redirects a decision while retaining a visible record of its earlier path." srcset="https://substackcdn.com/image/fetch/$s_!7urL!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png 424w, https://substackcdn.com/image/fetch/$s_!7urL!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png 848w, https://substackcdn.com/image/fetch/$s_!7urL!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png 1272w, https://substackcdn.com/image/fetch/$s_!7urL!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1a58f5a2-edbd-4b6f-86c6-2cf55f5f62c9_936x468.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h2>The ambition and the work we can do now</h2><p>Rich Sutton&#8217;s 2019 essay The Bitter Lesson asks us to look at how progress in AI has happened. His argument is that general methods such as search and learning become especially powerful as more computation becomes available [1]. That gives us a reason to care about a system&#8217;s ability to discover better ways of working. For a project assistant, we can ask what would allow its advice to improve after it encounters something its designers did not anticipate.</p><p>David Silver and Sutton take that ambition further in Welcome to the Era of Experience. They describe agents that act in the world, observe the consequences, and adapt through a continuing stream of experience [2]. Imagine an assistant that follows a project for months, checks what happened after its recommendations, and uses those outcomes to improve later decisions. That is the kind of experiential learning relevant to our discussion.</p><p>When dependable systems of this breadth will be available remains uncertain. Learning from experience already works in bounded settings, and Silver and Sutton include adaptation through context among its possible mechanisms [2]. The larger ambition involves sustaining useful learning across changing situations and long periods. More computing power may support that ambition, but a project system would still need access to relevant events and a meaningful way to judge its results. The big world hypothesis also cautions us about the idea of eventually learning everything. Javed and Sutton describe problems where the world is larger than an agent can fully represent [3]. Even a capable agent must select what to learn about its particular circumstances.</p><p>We can begin working on that problem with the models available to us. A model&#8217;s weights are the numerical parameters established through training. Those parameters can remain fixed while the surrounding system gives the model different information on its next attempt. If that information includes a useful lesson from an earlier outcome, it can change how the model approaches the task.</p><p>Reflexion, introduced by Shinn and colleagues, provides a concrete example [4]. An agent first attempts a task. An evaluator checks the attempt and provides feedback. A language model then examines the attempt alongside that feedback and writes a short reflection about what went wrong and what to try next. The system stores that reflection and supplies it as context for a later attempt [4].</p><p>The stored reflection gives the agent a specific basis for changing its approach. In the paper, this process supported improvements on several coding, reasoning, and simulated decision tasks, although results depended on the task and the quality of evaluation [4]. A mistaken evaluation can produce a misleading lesson. The process therefore needs feedback that is worth learning from.</p><p>For our illustrative project assistant, the same design idea would mean reviewing a missed start forecast, identifying the assumption behind it, and retaining a proposed check for next time. Applying this approach to projects would require its own testing. Its immediate value is that it makes learning a concrete system design concern. We must decide how outcomes become feedback, how feedback becomes a lesson, and how that lesson reaches a later decision.</p><h2>Give new evidence somewhere to go</h2><p>Return to the two teams. Suppose the assistant forecasts a Monday start for Team B after a budget approval. Monday arrives without an assigned crew. A project manager confirms that staffing requires a separate decision. Processing this event into a useful lesson takes several steps.</p><p>First, preserve enough of the event to understand the mistake. Record the forecast, the approval it relied on, and the actual outcome. Add the project manager&#8217;s explanation with its source and date. Keeping the observation and its interpretation separately makes the lesson easier to review. A missed start alone leaves several possible explanations open, including a staffing problem, a delivery delay, or a changed priority.</p><p>Next, propose a change in how the assistant should handle a similar decision. &#8220;Be more careful with approvals&#8221; gives little guidance. &#8220;For Team B, confirm staffing before treating budget approval as a start commitment&#8221; identifies both the relevant situation and the action to take. The strength of the lesson should match its evidence. One event might justify a provisional check. Confirmation of the team&#8217;s formal process would provide a stronger basis for making that check routine.</p><p>Then decide where the lesson belongs. Someone responsible for the workflow should check it against existing guidance and decide whether to accept it, revise it, or seek more evidence. The accepted lesson can become part of the instructions used when assessing Team B&#8217;s readiness. Keeping it attached to that team prevents a local observation from silently becoming a rule for every project.</p><p>The system must also bring the lesson back at the right moment. If it stays buried in a meeting transcript, a later forecast may overlook it. Context curation means selecting the relevant instructions and supporting evidence when the assistant prepares its answer. Here, that means supplying Team B&#8217;s staffing check alongside the new approval.</p><p>Finally, test what changed. Give the assistant a new Team B approval and see whether it asks about staffing. Give it a Team A case as well to check whether the lesson has created an unnecessary obstacle there. Track subsequent forecasts against actual starts, then revisit the instruction when the evidence or team process changes. A lesson needs a path through review, use, testing, and revision throughout its useful life.</p><h2>Let agents exchange lessons with their conditions</h2><p>Separate agents could maintain the working knowledge of each team and send relevant lessons to a coordinating agent. Each report would carry the observation, the proposed lesson, its supporting evidence, and the circumstances in which it applies.</p><p>Their reports may appear contradictory. Team A treats approval as readiness to start. Team B requires a staffing check. Preserving both accounts lets the coordinating agent ask what each approval actually commits the team to. Adopting either account as a universal rule would lose that distinction.</p><p>Transfer can hurt performance when knowledge from one setting is poorly suited to another. Machine learning research calls this negative transfer [5]. For our proposed project system, this suggests testing a shared lesson in the receiving team&#8217;s setting before allowing it to guide decisions there.</p><p>Consider a more capable agent with a large computing budget. It could spend resources improving a shared capability or send specialist agents to learn about particular domains. My view is that the useful choice depends on what is missing. Better reasoning may help all teams, while knowledge of local approval practices requires access to those teams. Specialists also create work for the parent agent, which must evaluate what comes back and decide what should remain local.</p><h2>Build the procedures that help the system learn</h2><p>Our practical opportunity today is to use AI models within procedures that improve as we work. </p><p>Each recurring task and specialty can have its own procedural context, a maintained account of how to approach the work, what to check, which evidence to consult, and when to ask for help.</p><p>A useful procedure develops through contact with the work. </p><p>The assistant applies it, people review its recommendations, and subsequent events provide evidence about what helped or failed. We can use that evidence to revise the instructions, keep the reasons for each revision, and test whether the next decision improves. Related tasks can share a common method while retaining the local details that change how it should be applied.</p><h2>Explain It to a Friend</h2><p>What if an organization gave every AI agent everything its colleagues had learned, then found that its advice became worse? One agent&#8217;s reliable shortcut could become another&#8217;s mistake. Bringing their experience together might erase the conditions that made each lesson useful. </p><p>Perhaps part of intelligence lies in preserving differences long enough to understand them. </p><p>A shared system would then need a way to keep disagreement available, even while coordinating action. What would we lose if every agent had to settle on the same account of how the world works? How would you explain this to a friend?</p><h2><strong>Notes</strong></h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p><strong>You are not receiving future PM Along AI posts simply by subscribing to EPM Research. Learn <a href="https://epmresearch.com/about#%C2%A7how-to-receive-posts-from-epm-sections">how to receive future PM Along AI posts</a>.</strong></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p>1. Sutton, R. S. (2019). <em><a href="https://bitterlesson.org/">The bitter lesson</a></em>.</p><p>2. Silver, D., &amp; Sutton, R. S. (2025). <em><a href="https://storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf">Welcome to the era of experience</a></em>.</p><p>3. Javed, K., &amp; Sutton, R. S. (2024). <em><a href="https://oaklab.ai/posts/the-big-world-hypothesis">The big world hypothesis and its ramifications for artificial intelligence</a></em>. Oak Lab.</p><p>4. Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., &amp; Yao, S. (2023). <em><a href="https://doi.org/10.48550/arXiv.2303.11366">Reflexion: Language agents with verbal reinforcement learning</a></em>. arXiv.</p><p>5. Wang, Z., Dai, Z., P&#243;czos, B., &amp; Carbonell, J. (2019). <em><a href="https://doi.org/10.48550/arXiv.1811.09751">Characterizing and avoiding negative transfer</a></em>. arXiv. Published at CVPR 2019.</p>]]></content:encoded></item><item><title><![CDATA[008 - Why Doesn’t a Larger AI Context Window Guarantee Better Answers in Project Management?]]></title><description><![CDATA[For complex projects, the challenge is not just how much AI can read, but how it discovers, connects, tests, and revises the evidence behind a decision.]]></description><link>https://epmresearch.com/p/008-why-doesnt-a-larger-ai-context</link><guid isPermaLink="false">https://epmresearch.com/p/008-why-doesnt-a-larger-ai-context</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Wed, 04 Nov 2026 21:52:27 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!lRrW!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Yes, it is true. A one-million-token context window can hold more than 700,000 words, roughly the scale of several Harry Potter novels [1]. </p><p>Imagine putting two Harry Potter novels beside two project risk management manuals and asking AI for a contingency recommendation. A capable model should recognize that the novels are irrelevant. But what if, <em>all four documents are technical, plausible, and potentially relevant</em>, but differ in authority, date, assumptions, and purpose.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!lRrW!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!lRrW!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png 424w, https://substackcdn.com/image/fetch/$s_!lRrW!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png 848w, https://substackcdn.com/image/fetch/$s_!lRrW!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png 1272w, https://substackcdn.com/image/fetch/$s_!lRrW!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!lRrW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png" width="1456" height="728" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:728,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:63621,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://epmresearch.com/i/223536247?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!lRrW!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png 424w, https://substackcdn.com/image/fetch/$s_!lRrW!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png 848w, https://substackcdn.com/image/fetch/$s_!lRrW!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png 1272w, https://substackcdn.com/image/fetch/$s_!lRrW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F060de821-9a7b-40ad-a8f1-25af8bc33ff2_1800x900.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Anyone who has built AI models or agents for real work will recognize the implication: <strong>a million-token context window does not remove the need to engineer the context</strong>. It matters especially when an agent must work across many projects, repeat decisions at scale, and deliver answers whose accuracy can be checked. </p><div class="pullquote"><p>The size of the window tells us how much information the model can accept, not whether that information supports the right conclusion.</p></div><p>As context windows grow, the greatest marginal gains may come from designing processes that help AI determine <em>which relationships, assumptions, and sources matter to a decision, and what additional evidence it needs to reach a reliable conclusion</em>.</p><h2>When every document looks relevant</h2><p>Imagine an AI agent estimating the contingency needed for a construction project. </p><p>It has access to risk procedures, engineering reports, and project schedules. However, a recent engineering report changes an earlier assumption, which affects the construction method, schedule, and costs. The AI might correctly read and cite every document but still reach the wrong conclusion if it fails to recognize how these changes are connected.</p><p>Project knowledge, as many other complex domins, is not merely a pile of documents or a flat table of facts. It is a network: <strong>a design choice depends on a site condition; the choice changes an activity; the activity affects resources and schedule; and those changes alter the risk picture</strong>. Some connections are explicit in project records. Others must be inferred and verified.</p><p>This is the problem we explored in the Uniform Project Ontology, that presents an ontology-based approach to representing industrial megaproject knowledge using linked data and the semantic web [2]. An ontology specifies entities and the meanings of relationships among them. A knowledge graph can express these entities as nodes and relationships as edges, supporting queries and reasoning across project information.</p><p>This does not mean relational databases are incapable of representing project relationships. Rather, the challenge is preserving their meaning across documents, systems, disciplines, and project phases. Nor does UPonto by itself establish that an LLM will answer more accurately. It explains <strong>why making project relationships explicit is a promising foundation for better AI context</strong>.</p><h2>Reading more, reasoning better</h2><p>There is experimental support for distinguishing context <em>capacity</em> from context <em>use</em>. </p><p>In <em>Lost in the Middle</em>, Liu and colleagues found that models could struggle to use relevant material placed in the middle of long inputs [3]. A later RAG study found that adding retrieved passages often improved performance at first, then degraded it as plausible but misleading passages accumulated [4]. The NoLiMa benchmark also showed that finding information becomes harder when a model must infer semantic connections rather than match similar words [5].</p><p>Nonetheless, sufficiently resourced long-context models outperform conventional retrieval-augmented generation, or RAG, on average in their tested tasks [6]. Some decisions genuinely require broad context because an unexpected relationship may be decisive.</p><p>So the question is not <em>How little information can we give the model?</em> It is <em>How can we ensure the information it uses is relevant, authoritative, appropriately connected, and sufficient?</em> </p><h2>The assumption hidden inside a convincing answer</h2><p>I see an analogy whenever I mark engineering exams. Students can be conscientious, understand the method, and perform each calculation carefully, yet reach the wrong conclusion because of one quiet assumption. </p><p>Two risks are treated as independent. A boundary condition is misunderstood. A design stage is taken for granted. The reasoning that follows may be internally consistent, but it is built on the wrong starting point.</p><p>We see something similar in AI-generated work. An agent may search diligently and still interpret a superseded document as the governing record. Worse, once it chooses that interpretation, it may keep retrieving evidence that appears to support it.</p><p>Language itself contributes to the difficulty. Clark and Brennan described communication as building <em>common ground</em>, the shared understanding that allows people to interpret each other&#8217;s words [7]. </p><p>Michael Polanyi&#8217;s work on tacit knowledge helps explain why professional judgment cannot always be fully captured in a manual [8]. A skilled worker, say a plumber or a barber learns through practice and feedback that builds <em>the important context</em>, not only through instructions. </p><h2>Why not let AI curate the context itself?</h2><p>A reasonable objection follows. If models are becoming better at planning, searching, and using tools, why should we do the curation ourselves? Let AI read the project repository, locate the right documents, and correct its own mistakes.</p><p>We should. At least, we should allow it to do much of the work.</p><p>Research on Self-RAG shows that models can be trained to decide when to retrieve evidence and to critique the information they obtain [9]. Anthropic&#8217;s guidance on context engineering likewise describes agents retrieving information as needed and refining their working context over successive steps [10]. Human experts do not need to hand-pick every page every time.</p><p>But an agent that can curate information is not necessarily an agent that can <em>validate its own assumptions</em>, at scale. </p><p>Research on intrinsic self-correction found cases where asking a model to revise its reasoning without external feedback failed or made performance worse [11]. This finding does not refute all methods of self-revision; it identifies a weakness in revision that introduces no independent evidence.</p><p>The useful distinction, then, is not between <strong>human-curated</strong> and <strong>AI-curated</strong> context. It is between a process that accepts its first interpretation and one that actively looks for reasons to change and optimize it.</p><div class="pullquote"><p>Context engineering is the process of governing the agent&#8217;s curation.</p></div><p>More and more, the mindset shift appears to be &#8220;how to interact with this new form of intelligence,&#8221; rather than, &#8220;how to use AI.&#8221;</p><h2>Explain it to a Friend</h2><p>What if an AI could read every document ever written about a project and still miss the one relationship that changes the decision? </p><p>Perhaps the deepest limit would not be memory, but judgment about what deserves attention and what remains unknown. A larger window would help it remember more. A better process would help it notice what it has misunderstood, seek evidence beyond its current interpretation, and sometimes ask a person who has seen the site. </p><p>Could the future of intelligence depend less on possessing all the answers than on knowing what to investigate next? </p><p>How would you explain this to a friend?</p><h2><strong>Notes</strong></h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p><strong><mark data-color="#fff2cc" style="background-color: rgb(255, 242, 204); color: rgb(0, 0, 0);">You are not receiving future PM Along AI posts simply by subscribing to EPM Research. Learn </mark><a href="https://epmresearch.com/about#&#167;how-to-receive-posts-from-epm-sections"><mark data-color="#fff2cc" style="background-color: rgb(255, 242, 204); color: rgb(0, 0, 0);">how to receive future PM Along AI posts</mark></a><mark data-color="#fff2cc" style="background-color: rgb(255, 242, 204); color: rgb(0, 0, 0);">.</mark></strong></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p><strong>[1] </strong><a href="https://blog.google/innovation-and-ai/products/google-gemini-next-generation-model-february-2024/">Google DeepMind. (2024). Introducing Gemini 1.5, Google&#8217;s next-generation AI model.</a></p><p><strong>[2] </strong><a href="https://doi.org/10.1016/j.aei.2020.101164">Zangeneh, P., &amp; McCabe, B. (2020). Ontology-based knowledge representation for industrial megaprojects analytics using linked data and the semantic web. Advanced Engineering Informatics, 46, 101164.</a></p><p><strong>[3] </strong><a href="https://doi.org/10.1162/tacl_a_00638">Liu, N. F., et al. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157&#8211;173.</a></p><p><strong>[4] </strong><a href="https://proceedings.iclr.cc/paper_files/paper/2025/hash/5df5b1f121c915d8bdd00db6aac20827-Abstract-Conference.html">Jin, B., Yoon, J., Han, J., &amp; Arik, S. (2025). Long-context LLMs meet RAG: Overcoming challenges for long inputs in RAG. ICLR.</a></p><p><strong>[5] </strong><a href="https://proceedings.mlr.press/v267/modarressi25a.html">Modarressi, A., et al. (2025). NoLiMa: Long-context evaluation beyond literal matching. Proceedings of the 42nd International Conference on Machine Learning.</a></p><p><strong>[6] </strong><a href="https://doi.org/10.18653/v1/2024.emnlp-industry.66">Li, Z., Li, C., Zhang, M., Mei, Q., &amp; Bendersky, M. (2024). Retrieval augmented generation or long-context LLMs? A comprehensive study and hybrid approach. EMNLP Industry Track, 881&#8211;893.</a></p><p><strong>[7] </strong><a href="https://doi.org/10.1037/10096-006">Clark, H. H., &amp; Brennan, S. E. (1991). Grounding in communication. In Perspectives on socially shared cognition (pp. 127&#8211;149). APA.</a></p><p><strong>[8] </strong><a href="https://press.uchicago.edu/ucp/books/book/chicago/T/bo6035368">Polanyi, M. (1966). The tacit dimension. Doubleday.</a></p><p><strong>[9] </strong><a href="https://proceedings.iclr.cc/paper_files/paper/2024/hash/25f7be9694d7b32d5cc670927b8091e1-Abstract-Conference.html">Asai, A., Wu, Z., Wang, Y., Sil, A., &amp; Hajishirzi, H. (2024). Self-RAG: Learning to retrieve, generate, and critique through self-reflection. ICLR.</a></p><p><strong>[10] </strong><a href="https://www.anthropic.com/research/effective-context-engineering-for-ai-agents">Anthropic Engineering. (2025). Effective context engineering for AI agents.</a></p><p><strong>[11] </strong><a href="https://proceedings.iclr.cc/paper_files/paper/2024/hash/8b4add8b0aa8749d80a34ca5d941c355-Abstract-Conference.html">Huang, J., et al. (2024). Large language models cannot self-correct reasoning yet. ICLR.</a></p>]]></content:encoded></item><item><title><![CDATA[007 - The Case for Hybrid Local and Frontier AI: Sensitive Project Management Data]]></title><description><![CDATA[Keep sensitive project files on a machine you control, send frontier models only cleaned questions, and use that private space to experiment.]]></description><link>https://epmresearch.com/p/009-the-case-for-hybrid-local-and</link><guid isPermaLink="false">https://epmresearch.com/p/009-the-case-for-hybrid-local-and</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Thu, 29 Oct 2026 22:30:49 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!s5VB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Imagine a claims team at a general contractor preparing a delay claim. </p><p>The evidence sits in daily logs, RFIs, schedule updates, subcontractor quotes and two years of email. A frontier model would probably reason through that material better than anything else available. It is also the last place the commercial lead wants those files to go.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!s5VB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!s5VB!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg 424w, https://substackcdn.com/image/fetch/$s_!s5VB!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg 848w, https://substackcdn.com/image/fetch/$s_!s5VB!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg 1272w, https://substackcdn.com/image/fetch/$s_!s5VB!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!s5VB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg" width="1456" height="728" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/ccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:728,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:4172,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/svg+xml&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://epmresearch.com/i/222708055?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!s5VB!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg 424w, https://substackcdn.com/image/fetch/$s_!s5VB!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg 848w, https://substackcdn.com/image/fetch/$s_!s5VB!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg 1272w, https://substackcdn.com/image/fetch/$s_!s5VB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fccaa9356-76ad-425c-993a-6007463f6e12_1200x600.svg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>That tension is common. </p><p>When construction practitioners were asked to rank the challenges of adopting generative AI on projects, data privacy, security and legal liability came first, ahead of poor training data and the lack of construction-specific knowledge [1]. </p><p>A 2026 systematic review of 142 studies on large language models in construction reached a similar point: projects hold drawings, BIM models, contracts and operational records, and relying on cloud-hosted models raises the risk of leakage and unauthorized access [2]. </p><p>Outside construction, Cisco&#8217;s 2024 survey of 2,600 privacy and security professionals found that 27% of organizations had banned generative AI at least temporarily, while 48% admitted that non-public company information had been entered into these tools anyway [3].</p><p>So the practical question for a project organization is rarely &#8220;frontier or local?&#8221; It is closer to &#8220;which part of this work needs the frontier, and which part should stay home?&#8221;</p><h2>The gap is real, but it is measured in months</h2><p>Frontier models are better, and the race at the top is hard to follow. </p><p>Still, Epoch AI&#8217;s capability index shows the best open-weight models, the ones you can download and run yourself, trailing the best closed models by about four months on average since January 2026 [4]. </p><p>That gap matters most for the hardest reasoning. Many project tasks are different: reading a specification section, extracting dates from correspondence, classifying defects, drafting a first summary. For those, a model a few months behind the frontier may well be good enough, and a team can test that on its own documents.</p><p>Construction researchers have already started to split the work this way. Jeon and Lee built a defect question-answering system in which a closed cloud model generated the training data, while a fine-tuned open-source model handled the sensitive defect records. Their hybrid version scored highest of the approaches they compared [5]. </p><p>Singh and Hsieh used commercial models to generate and augment construction schedules, then used small, locally deployed models to evaluate the results, so sensitive data did not need to leave a secure environment [6].</p><h2>How a hybrid workflow handles a sensitive question</h2><p>Here is the idea applied to the claims example. The steps below are an illustrative workflow, not a tested product.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!iCqD!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!iCqD!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png 424w, https://substackcdn.com/image/fetch/$s_!iCqD!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png 848w, https://substackcdn.com/image/fetch/$s_!iCqD!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png 1272w, https://substackcdn.com/image/fetch/$s_!iCqD!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!iCqD!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png" width="901" height="541" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/bd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:541,&quot;width&quot;:901,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Title: Hybrid local and frontier workflow - Description: Flow diagram: project files stay on a controlled machine where a local model reads and redacts them; only a clean question goes to a frontier model; the answer returns in placeholders, the local model restores details and checks sources, and a person approves.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Title: Hybrid local and frontier workflow - Description: Flow diagram: project files stay on a controlled machine where a local model reads and redacts them; only a clean question goes to a frontier model; the answer returns in placeholders, the local model restores details and checks sources, and a person approves." title="Title: Hybrid local and frontier workflow - Description: Flow diagram: project files stay on a controlled machine where a local model reads and redacts them; only a clean question goes to a frontier model; the answer returns in placeholders, the local model restores details and checks sources, and a person approves." srcset="https://substackcdn.com/image/fetch/$s_!iCqD!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png 424w, https://substackcdn.com/image/fetch/$s_!iCqD!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png 848w, https://substackcdn.com/image/fetch/$s_!iCqD!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png 1272w, https://substackcdn.com/image/fetch/$s_!iCqD!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbd80b8ed-538b-4c14-b782-dc3df0c843b7_901x541.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p style="text-align: center;"><em>An illustrative hybrid workflow. The sensitive reading happens locally; the frontier model sees only a cleaned question.</em></p><ul><li><p>The files stay on a machine the organization controls. A local model reads them, finds the relevant events and builds a timeline.</p></li><li><p>Before anything leaves, the local model replaces sensitive specifics with placeholders. Party names become &#8220;Owner&#8221; and &#8220;Sub A&#8221;, dollar values become ranges, and project identifiers disappear.</p></li><li><p>Only the cleaned question goes to the frontier model, for example: &#8220;Given this sequence of notices and delays under a typical stipulated-price contract, what are the strongest and weakest points of an excusable-delay argument?&#8221;</p></li><li><p>The answer comes back in placeholders. The local model maps it to the real records and flags any point it cannot trace to a source document.</p></li><li><p>A person reviews and approves the result.</p></li></ul><p>Each model does what it does best. The frontier model contributes reasoning about structure; the local model does the reading, which is where the sensitive volume lives. Research on this local-remote pattern supports the economics too. In the Minions study, a small on-device model read long financial, medical and scientific documents while a frontier cloud model broke the task into smaller subtasks. The best protocol recovered 97.9% of the frontier model&#8217;s accuracy at about 18% of its cloud cost [7]. That study measured cost and quality, not privacy or construction work, but it shows the frontier model does not need to read everything to contribute.</p><p>Two qualifications matter. Redaction is imperfect: a timeline with names removed can still identify a project to someone who knows it, so decide what is too revealing even in placeholder form. And local is not automatically secure. A desk machine still needs access control, encryption and updates. The benefit is that your organization decides where the data goes, not that the risk disappears.</p><h2>A private machine is also a laboratory</h2><p>The second argument for local AI is less about secrecy and more about learning.</p><p>Adopting AI in project work takes a lot of trial and error on real data. Which instruction extracts change-order dates reliably? Does a model read your company&#8217;s daily-log format correctly? How often does it misread a specification reference? Those questions are best answered on your own past projects, which are exactly the files people hesitate to upload.</p><p>A machine on the desk lowers the cost of asking. A team can run the same task a hundred times on a closed project, compare models and keep every output for review, with no per-token bill and no approval needed for each upload. Mistakes made while testing stay inside the building.</p><h2>What a desk-sized AI computer actually buys</h2><p>Aside from custom AI rigs, two machines currently define the high end of local AI hardware, and they make different bets. Three numbers explain the difference:</p><ul><li><p><strong>Memory</strong> decides how large a model fits. At 4-bit precision, a rough rule is half a byte per parameter, so a 400-billion-parameter model needs about 200 GB before any working space.</p></li><li><p><strong>Memory bandwidth</strong> decides how fast the answer appears. Generating each word means reading the model&#8217;s active weights, so more bandwidth gives faster output.</p></li><li><p><strong>Compute</strong> decides how fast long inputs are read, how quickly a model can be fine-tuned and how many people can share the machine.</p></li></ul><p>Apple&#8217;s new Mac Studio with the M5 Ultra goes after the first two. It offers up to 512 GB of unified memory with 1.2 TB/s of bandwidth, Apple claims up to 4.3 times the peak AI compute of the previous M3 Ultra, and several machines can be clustered over Thunderbolt 5 [8]. The M5 Ultra starts at US$5,499; the 512 GB configuration went on sale this month at $22,299 [8,9].</p><p>NVIDIA&#8217;s DGX Spark goes after the third. Its GB10 chip is rated at up to 1 petaFLOP of 4-bit AI compute (a theoretical figure that assumes sparsity), paired with 128 GB of memory at 273 GB/s [10]. It runs the same CUDA software stack used in NVIDIA&#8217;s data centers, which matters if a prototype will later move to larger infrastructure. Memory shortages have pushed its price from $3,999 at launch to $6,950, and NVIDIA added a 64 GB version at $4,999 this month [11].</p><p>The benchmarks follow the bandwidth. Tom&#8217;s Hardware measured the M5 Ultra generating text almost four times faster than the DGX Spark on a dense 27-billion-parameter model, with faster prompt processing as well [12]. LMSYS&#8217;s review of the Spark shows the other side of the trade: a 70-billion-parameter model processed its prompt at 803 tokens per second but generated only 2.7 tokens per second, which the reviewers judged better suited to prototyping than production [13]. The Spark&#8217;s total throughput rises sharply when many requests run at once, and it can fully fine-tune an 8-billion-parameter model on the desk [10].</p><p>So the competition is between capacity and generation speed on one side, and compute and ecosystem continuity on the other. A reasonable reading for project teams: a large-memory Mac suits a few people running big open models interactively; a Spark suits teams that want to fine-tune, serve many small requests, or build something they later deploy on NVIDIA servers. Neither replaces the frontier. Both make the local half of a hybrid workflow practical.</p><h2>A small experiment to start</h2><p>You do not need the most expensive machine to begin. Pick one closed project and one repeatable task, such as extracting delay events from daily logs. Run it with a local open model on the original files, and with a frontier model on a redacted version. Record accuracy against an answer a person prepared, the time spent correcting each output, and anything the redaction failed to hide. That single comparison will tell you more about where your organization&#8217;s line between local and cloud belongs than any benchmark.</p><h2>Explain It to a Friend</h2><p>What if every question your team asked AI were visible to your competitors? People would ask fewer and safer questions, and the organization would learn less. </p><p>What a team is willing to show limits what it can learn. </p><p>Local AI moves that limit: it gives a team a private room where clumsy questions, failed prompts and half-formed ideas cost nothing. If privacy is what makes honest trial and error possible, then experimentation is the natural way into this new domain, not something to wait on until the tools mature. So cheers to experimentation, done somewhere safe enough to be wrong. How would you explain this to a friend?</p><h2><strong>Notes</strong></h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p>Subscribe to EPM Research and learn <a href="https://epmresearch.com/about#%C2%A7how-to-receive-posts-from-epm-sections">how to receive future PM Along AI posts</a>.</p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p>[1] Fern&#225;ndez, A. V., &amp; Shen, Y. (2026). <a href="https://doi.org/10.1080/15623599.2026.2664476">Prioritising generative AI adoption challenges in construction projects: A Fuzzy Analytic Hierarchy Process approach</a>. International Journal of Construction Management.</p><p>[2] Gao, Y., Yiu, T. W., Shen, X., &amp; Tam, V. W. Y. (2026). <a href="https://doi.org/10.1108/ECAM-10-2025-1668">Large language models in smart construction: A systematic review of implementation strategies, applications and future directions</a>. Engineering, Construction and Architectural Management, 33(15), 159&#8211;181.</p><p>[3] Cisco. (2024, January 25). <a href="https://newsroom.cisco.com/c/r/newsroom/en/us/a/y2024/m01/organizations-ban-use-of-generative-ai-over-data-privacy-security-cisco-study.html">More than 1 in 4 organizations banned use of GenAI over privacy and data security risks: New Cisco study</a>. Cisco Newsroom.</p><p>[4] Edwards, J., &amp; Emberson, L. (2026, May 29). <a href="https://epoch.ai/data-insights/open-closed-eci-gap">Open models lag state-of-the-art closed models by 4 months</a>. Epoch AI.</p><p>[5] Jeon, K., &amp; Lee, G. (2025). <a href="https://doi.org/10.1016/j.aei.2024.103076">Hybrid large language model approach for prompt and sensitive defect management: A comparative analysis of hybrid, non-hybrid, and GraphRAG approaches</a>. Advanced Engineering Informatics, 64, 103076.</p><p>[6] Singh, A. K., &amp; Hsieh, S.-H. (2026). <a href="https://doi.org/10.1016/j.aei.2025.103825">Multi-LLM-based augmentation and synthetic data generation of construction schedules and task descriptions with SLM-as-a-judge assessment</a>. Advanced Engineering Informatics, 69, 103825.</p><p>[7] Narayan, A., Biderman, D., Eyuboglu, S., May, A., Linderman, S., Zou, J., &amp; R&#233;, C. (2025). <a href="https://proceedings.mlr.press/v267/narayan25a.html">Cost-efficient collaboration between on-device and cloud language models</a>. Proceedings of the 42nd International Conference on Machine Learning, PMLR 267, 45682&#8211;45719.</p><p>[8] Apple. (2026, August 25). <a href="https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/">Apple introduces new Mac Studio with M5 Max and M5 Ultra</a>. Apple Newsroom.</p><p>[9] VideoCardz. (2026, October). <a href="https://videocardz.com/newz/apple-mac-studio-with-512gb-ram-now-costs-22299-memory-upgrade-alone-is-15500">Apple Mac Studio with 512GB RAM now costs $22,299, memory upgrade alone is $15,500</a>. VideoCardz.com.</p><p>[10] Jain, D. (2025, October 14). <a href="https://www.storagereview.com/review/nvidia-dgx-spark-review-the-ai-appliance-bringing-datacenter-capabilities-to-desktops">NVIDIA DGX Spark review: The AI appliance bringing datacenter capabilities to desktops</a>. StorageReview.</p><p>[11] The Register. (2026, October). <a href="https://www.theregister.com/a/5300622">Nvidia debuts $4,999 DGX Spark with half the RAM and storage, amid memory crunch</a>. The Register.</p><p>[12] Freedman, A. E. (2026, September 21). <a href="https://www.tomshardware.com/desktops/mini-pcs/apple-mac-studio-m5-ultra-review">Apple Mac Studio (M5 Ultra) review: Local model citizen outpaces DGX Spark and Threadripper</a>. Tom&#8217;s Hardware.</p><p>[13] LMSYS Org. (2025, October 13). <a href="https://www.lmsys.org/blog/2025-10-13-nvidia-dgx-spark">NVIDIA DGX Spark in-depth review: A new standard for local AI inference</a>. LMSYS Blog.</p>]]></content:encoded></item><item><title><![CDATA[006 - How to Build a Simple AI Scheduled Task of Review and Sending Emails with Improvable Structure?]]></title><description><![CDATA[Keep the procedure in editable files, reuse its skills, and let a weekly schedule run separate improvable nodes in a workflow.]]></description><link>https://epmresearch.com/p/006-how-to-build-a-simple-ai-scheduled</link><guid isPermaLink="false">https://epmresearch.com/p/006-how-to-build-a-simple-ai-scheduled</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Sat, 24 Oct 2026 17:47:31 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!UVPS!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Every Friday, you open the project inbox, look for commitments that need an update, send reminders, and prepare a summary. </p><p>This is the simple process that allows you to turn that routine into a small, organized set of files that an AI application reads and conduct each week. </p><p>The useful design choice is to separate the overall procedure into steps and separate skills it uses. Changing the procedure then becomes a matter of editing a file, while the weekly schedule continues to point to it.</p><p>We will build an illustrative weekly email review that maintains an action list, sends reminders and requests for updates, and saves a summary. Its follow-up is complete when the email service confirms the message was sent.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!UVPS!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!UVPS!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg 424w, https://substackcdn.com/image/fetch/$s_!UVPS!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg 848w, https://substackcdn.com/image/fetch/$s_!UVPS!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg 1272w, https://substackcdn.com/image/fetch/$s_!UVPS!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!UVPS!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg" width="1456" height="728" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:728,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1277,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/svg+xml&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://epmresearch.com/i/221939896?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!UVPS!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg 424w, https://substackcdn.com/image/fetch/$s_!UVPS!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg 848w, https://substackcdn.com/image/fetch/$s_!UVPS!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg 1272w, https://substackcdn.com/image/fetch/$s_!UVPS!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F767eec17-728b-45fb-84aa-1abb5f448d62_1200x600.svg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h2>Start with an application that can do the work</h2><p>To begin, ask the AI you&#8217;re using the following prompt to check the following four capabilities separately.</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;08af4147-eb74-40e7-be4f-a834adec5a5f&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">I want to set up a weekly scheduled task. For each of the four capabilities below, reply with Yes or No (can you do it now, or can you get it by installing an available plugin or connector?). After each answer, give one line on the path forward. Check my actual connectors and plugins before answering. Keep the whole reply under 120 words.

1- Read emails
2- Read and write a cloud folder
3- Send emails
4- Run on a weekly schedule</code></pre></div><p>Many organizations now have not opened access to AI models to do these four. </p><p>If you are part of such organization, you may create a simple parallel workflow using a disposable Gmail account and your favorite AI model. But please use this simple post to remind the policy makers in your organization how soon the need to act. </p><p>For example, ChatGPT and Claude scheduled tasks can use available plugins and skills to connect to Gmail  and Google drive. The process workflow needs to be on cloud, otherwise tasks needing local files will depend on the computer being online and available. [1,2]</p><p>Automatic email requires a sending tool and a permission mode that permits unattended use. Connecting an inbox alone does not settle that question. </p><p>Anthropic&#8217;s Gmail documentation, for example, describes approval for sending by default and organization controls for allowing actions without asking each time. Check the behavior of your own setup before scheduling it. [3] You can further add a human permission step before sending emails as well.</p><h2>Give the routine a home</h2><p>Create a folder called Weekly Project Review in accessible cloud storage (Google drive). Ask AI to create the files shown below. Keeping the procedure, working records and outputs together makes the routine easier to inspect and revise.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!e3KL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!e3KL!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png 424w, https://substackcdn.com/image/fetch/$s_!e3KL!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png 848w, https://substackcdn.com/image/fetch/$s_!e3KL!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png 1272w, https://substackcdn.com/image/fetch/$s_!e3KL!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!e3KL!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png" width="995" height="535" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:535,&quot;width&quot;:995,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Annotated task folder map showing agent.md, Skills/write-follow-up/SKILL.md, actions.md, Reports and sent-log.md, with each role explained.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Annotated task folder map showing agent.md, Skills/write-follow-up/SKILL.md, actions.md, Reports and sent-log.md, with each role explained." title="Annotated task folder map showing agent.md, Skills/write-follow-up/SKILL.md, actions.md, Reports and sent-log.md, with each role explained." srcset="https://substackcdn.com/image/fetch/$s_!e3KL!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png 424w, https://substackcdn.com/image/fetch/$s_!e3KL!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png 848w, https://substackcdn.com/image/fetch/$s_!e3KL!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png 1272w, https://substackcdn.com/image/fetch/$s_!e3KL!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F784c7b14-d2fd-494b-a5bb-44a2f4642145_995x535.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The .md extension means a Markdown text file. Markdown uses simple symbols for structure: # introduces a heading and a hyphen can introduce a list item. You can read the words in an ordinary text editor, and ask AI to revise them. A Markdown file supplies instructions; the AI application follows those instructions using its tools. [4]</p><p>On each run, the AI reads the current instructions and working records in the task folder. Each file or folder has a specific role:</p><ul><li><p><code>Weekly Project Review/</code> is the main task folder. It keeps the routine&#8217;s instructions, skills, working records, and outputs together.</p></li><li><p><code>agent.md</code> describes the overall procedure: which emails to review, how to update the action list, when to send follow-ups, and what results to save.</p></li><li><p><code>Skills/</code> holds reusable instructions for particular parts of the work. Each skill has its own subfolder, such as <code>write-follow-up/</code>.</p></li><li><p><code>Skills/write-follow-up/SKILL.md</code> explains how to write reminders and requests for updates, including their tone, length, and required information. Other routines can reference this same skill.</p></li><li><p><code>actions.md</code> maintains the ongoing action list, including each commitment, its owner, due date, and source email. It carries relevant commitments forward into the next weekly review.</p></li><li><p><code>Reports/</code> stores dated weekly summaries so you can review what changed, which follow-ups were sent, and which items still need attention.</p></li><li><p><code>sent-log.md</code> records confirmed email sends, including the related action, recipient, sending time, and message reference. The routine checks this record to help prevent duplicate follow-ups.</p></li></ul><h2>Separate the procedure from the writing skill</h2><p>Put the overall process in agent.md. </p><p>It identifies what to review, which records to update, which skill to use, and what to save. A short version could read:</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;86f3e05d-46a6-4a86-b432-c99aabc7e3d2&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext"># Weekly project email review

1. Read actions.md and sent-log.md.

2. Review the past seven days of project [Placeholders] emails.

3. Update recorded commitments, owners and dates from source threads.

4. Read Skills/write-follow-up/SKILL.md and use it to write reminders and requests for missing updates.

5. Send authorized follow-ups to confirmed owners. Check the log and sent mail to avoid duplicates. Record confirmed sends with their message references.

6. Save the updated records and a dated summary in Reports/. Include any step that could not be completed.</code></pre></div><p>Here, agent.md is the filename we explicitly tell the scheduled AI to read. </p><p>The process can read a spreadsheet instead of last week emails. That can also be updated as well.</p><p>Filename conventions vary. Codex, for example, automatically discovers AGENTS.md under its documented rules. In another application, naming a file agent.md may give it no special behavior until you reference it. [5]</p><p>Put the email-writing method in Skills/write-follow-up/SKILL.md. A skill is a reusable set of instructions, sometimes accompanied by references, templates or scripts. Compatible tools use its name and description to help identify it. For this example, the instructions can stay short: [6]</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;markdown&quot;,&quot;nodeId&quot;:&quot;4bb2c080-369d-467b-8a3c-26b639ee0bba&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-markdown">---

name: write-follow-up

description: Write concise project reminders and update requests.

---

# Write a follow-up

- Address the confirmed action owner.

- Identify the commitment and its recorded due date.

- Ask for a status update or the specific missing information.

- Use a professional tone and no more than 120 words.

- Keep to the recorded commitment and deadline.

- Preserve the source thread when appropriate.</code></pre></div><p>This separation gives you two ways to improve the routine. Edit agent.md to change the review process. Edit SKILL.md to change how follow-ups are written.</p><div class="pullquote"><p>This algorithmic division of processes and skills is the  mindset shift you need to expand this process to much more complex tasks.</p></div><p>Another routine, such as procurement follow-ups, can reference that same skill file by its accessible path. Sharing the file lets an improvement serve both routines; separate copies need separate updates.</p><h2>Keep a record of what happened</h2><p>Suppose a source email records an owner promising a revised lookahead plan by Tuesday. The weekly review finds no update and uses the writing skill to request one. After the email tool confirms sending, the routine records the action identifier, recipient, time and message reference. It marks the follow-up as sent and retains the project action in the ongoing list.</p><p>That record matters when a run is repeated. The routine can check its log and sent mail before sending another reminder for the same action and review period. The weekly summary should say what changed, which emails were sent, and which items still need attention. If sending or saving fails, it should report the unfinished step.</p><h2>Test once and then schedule</h2><p>Run the routine manually before scheduling it. Review the chosen source emails and proposed messages, test saving and reading back a report, and use yourself as a test recipient. Check that a confirmed send appears in the record. OpenAI&#8217;s scheduling guidance similarly recommends testing the prompt manually and reviewing the first scheduled results. [1]</p><p>Then create the weekly schedule with a short instruction pointing to the procedure:</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;37597ad4-d4bf-491d-9211-95e35b4bb526&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Every [weekday] at [time and time zone], open [cloud folder]/agent.md, 
read its current instructions and the skills it references, and carry out the weekly review. 
Use the saved action list and sending record. 
Save the summary and report any step that could not be completed.</code></pre></div><p>The schedule now identifies when to start and where to find the method. To adjust reminder wording next month, update the shared writing skill. To change which project emails are reviewed, update agent.md. The next run must retrieve those current files for the changes to take effect.</p><h2>Explain It to a Friend</h2><p>What if you replaced the AI model next month, say from Gemini to GPT, but kept every procedure, skill and record? The new system might need different connections, yet it would inherit a description of what you wanted done and how you judged a useful result. </p><p>This suggests that part of a team&#8217;s capability lives outside any individual operator. It lives in the methods people leave behind. The files do not contain all the judgment needed to act well, but they make some of that judgment available to someone, or something, new. How would you explain this to a friend?</p><h2>Notes</h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future. </p><p>Subscribe to EPM Research and learn <a href="https://epmresearch.com/about#%C2%A7how-to-receive-posts-from-epm-sections">how to receive future PM Along AI posts</a>.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p>[1] OpenAI. (n.d.). <a href="https://learn.chatgpt.com/docs/automations">Scheduled tasks.</a></p><p>[2] Anthropic. (n.d.). <a href="https://support.claude.com/en/articles/13854387-schedule-recurring-tasks-in-claude-cowork">Schedule recurring tasks in Claude Cowork.</a></p><p>[3] Anthropic. (n.d.). <a href="https://support.claude.com/en/articles/10166901-use-google-workspace-connectors">Use Google Workspace connectors.</a></p><p>[4] CommonMark. (n.d.). <a href="https://commonmark.org/help/">Markdown reference.</a></p><p>[5] OpenAI. (n.d.). <a href="https://learn.chatgpt.com/docs/agent-configuration/agents-md">Custom instructions with AGENTS.md.</a></p><p>[6] OpenAI. (n.d.). <a href="https://developers.openai.com/plugins/concepts/skills">Skills.</a></p>]]></content:encoded></item><item><title><![CDATA[005 - Ask AI What Human Input Could Improve Its Work?]]></title><description><![CDATA[Identify the smallest useful human contribution, connect it to a specific weakness, and verify its effect on the next version.]]></description><link>https://epmresearch.com/p/005-ask-ai-what-human-input-could</link><guid isPermaLink="false">https://epmresearch.com/p/005-ask-ai-what-human-input-could</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Fri, 16 Oct 2026 14:03:27 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!CxRn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Imagine reviewing a construction schedule narrative. The draft reads well, yet two supplied documents give different completion dates. Asking AI to &#8220;improve the narrative&#8221; may produce cleaner prose. It cannot establish which date is approved.</p><p>The better question is: what is the smallest thing I could provide that would materially improve this work? A useful answer identifies a particular document, observation, example, or decision and explains what would change once it is available.</p><p>This shifts human involvement from reviewing every sentence to addressing a specific gap. It does not replace professional review. It makes one part of that review more deliberate.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!CxRn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!CxRn!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png 424w, https://substackcdn.com/image/fetch/$s_!CxRn!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png 848w, https://substackcdn.com/image/fetch/$s_!CxRn!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png 1272w, https://substackcdn.com/image/fetch/$s_!CxRn!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!CxRn!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png" width="760" height="380" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:380,&quot;width&quot;:760,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;An autonomous AI work cycle crosses the boundary to request a targeted human contribution, then returns to verification.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="An autonomous AI work cycle crosses the boundary to request a targeted human contribution, then returns to verification." title="An autonomous AI work cycle crosses the boundary to request a targeted human contribution, then returns to verification." srcset="https://substackcdn.com/image/fetch/$s_!CxRn!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png 424w, https://substackcdn.com/image/fetch/$s_!CxRn!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png 848w, https://substackcdn.com/image/fetch/$s_!CxRn!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png 1272w, https://substackcdn.com/image/fetch/$s_!CxRn!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa563e804-14d3-4dc9-aa18-04e0755809b0_760x380.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h2>Ask for a contribution with a purpose</h2><p>Before asking you, AI should check the information and approved tools it already has. If a drawing revision, calculation, or baseline is available, the system should retrieve it rather than ask you to upload the whole archive. Anthropic&#8217;s context-engineering guidance favors relevant, high-signal information over indiscriminate accumulation of context [1].</p><p>Where something really is missing, the request should connect four things: the gap, the contribution, why the gap matters, and the check the contribution would enable. An AI-generated request is still a hypothesis about what is needed, not proof that it has found every weakness.</p><h2>What should AI do before interrupting you?</h2><p>Not every uncertainty justifies stopping work. The following illustrated decision path separates work AI can resolve independently from cases that require human input. It is a proposed workflow for project work, not a validated algorithm.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!RZz3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!RZz3!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png 424w, https://substackcdn.com/image/fetch/$s_!RZz3!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png 848w, https://substackcdn.com/image/fetch/$s_!RZz3!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png 1272w, https://substackcdn.com/image/fetch/$s_!RZz3!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!RZz3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png" width="936" height="404" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:404,&quot;width&quot;:936,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Flow diagram: AI identifies a gap. If approved sources resolve it, retrieve and verify. If not material, record an optional request and continue. If material, halt if essential, ask for a targeted contribution, check the response, and reverify.&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Flow diagram: AI identifies a gap. If approved sources resolve it, retrieve and verify. If not material, record an optional request and continue. If material, halt if essential, ask for a targeted contribution, check the response, and reverify." title="Flow diagram: AI identifies a gap. If approved sources resolve it, retrieve and verify. If not material, record an optional request and continue. If material, halt if essential, ask for a targeted contribution, check the response, and reverify." srcset="https://substackcdn.com/image/fetch/$s_!RZz3!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png 424w, https://substackcdn.com/image/fetch/$s_!RZz3!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png 848w, https://substackcdn.com/image/fetch/$s_!RZz3!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png 1272w, https://substackcdn.com/image/fetch/$s_!RZz3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F83ff60f9-18e7-444c-9a15-a27c1f82eab0_936x404.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p style="text-align: center;"><em>Deciding when to retrieve, continue, or ask for human input.</em></p><p>Return to the schedule example. If the current approved baseline is already available, retrieve it and check the narrative. If neither date is authoritative, ask for the signed baseline revision or the planner&#8217;s documented decision. If the inconsistency affects an accepted milestone, stop rather than select the more plausible date. Afterward, verify the revised narrative against the approved record and check each affected milestone.</p><p>The same principle applies to quality, not only factual correctness. If a technically accurate report misses the audience&#8217;s needs, one annotated paragraph from an approved example may communicate the expected level of detail. That example guides presentation; current project evidence must still support the facts.</p><h2>A prompt you can reuse</h2><p>Paste the task or work product, its approved sources, and any verification findings. This prompt asks for a small, consequential contribution rather than a general list of things the AI would like to know.</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;65bccfda-90ec-4670-b7df-2827be04560d&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Review this work product, its requirements, approved sources, and any verification findings.

What is the smallest contribution I could provide that would most improve its accuracy or usefulness? First use information and tools already available within your permissions.

For each material request, identify the specific gap, why it matters, exactly what I should provide, and how you will check whether it helped. Prioritize benefit against the effort required from me. 

Separate essential requests from optional improvements. Do not invent missing facts or numerical benefits.

Ask me for the highest-value contribution first. State what can continue without it and what must stop. If none is needed, say so.

After I respond, check the response against approved evidence, flag conflicts, and reverify the affected work. Report what improved and what remains unresolved.</code></pre></div><h2>Connect the request to the improvement loop</h2><p>This works on a single draft, but it also fits a bounded improvement loop. Resolve essential gaps before starting. During iterations, record optional requests without repeatedly interrupting the reviewer. At a material unresolved finding, halt. At the agreed plateau, return the best retained result and one specific request.</p><p>For an existing loop, this companion instruction makes the handoff explicit:</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;4ef3bdd2-2724-41da-8e75-9a905f0af101&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Before the run, resolve essential input; log optional requests without interruption. 

At a material UNRESOLVED finding, HALT. 

At the agreed plateau, return the best retained version and one targeted request. 

Check the response and reverify. 

If fixed controls change, obtain approval and begin a new bounded run.</code></pre></div><p>A response that changes approved sources, acceptance criteria, or verification rules needs authorization and a new baseline. Otherwise, proceed only within the remaining approved scope and budget.</p><p>This is consistent with human-AI design research recommending easy correction and clarification when goals are uncertain [2]. The aim is not more conversation, but a purposeful intervention whose effect can be checked.</p><h2>How do you know the human input helped?</h2><p>Compare before and after using the same acceptance conditions where possible. Did the missing record arrive? Was the conflict resolved? Did the required technical or communication quality improve? More fluent wording or a more confident AI answer does not establish improvement.</p><p>NIST&#8217;s AI Risk Management Framework emphasizes input from domain experts and documentation of measurable improvements or declines [3]. </p><p>For consequential project work, preserve the original finding, the source and authority of the human contribution, the action taken, and the verification result. If the acceptance conditions change, avoid comparing the new result as though the original benchmark were unchanged.</p><h2>Explain It to a Friend</h2><p>What if AI became extraordinarily capable at solving problems, but could never be certain that it had noticed the one missing fact that mattered? Even an excellent answer could rest on an invisible gap. </p><p>Perhaps part of intelligence is not only knowing how to answer, but recognizing when an answer depends on something outside one&#8217;s knowledge. </p><p>That makes human collaboration more crucial than correcting errors. The person might supply the missing evidence, but they might also recognize a question AI never thought to ask. Knowing what we do not know might become a shared skill. How would you explain this to a friend?</p><h2>Notes</h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future. </p><p>Subscribe to EPM Research and learn <a href="https://epmresearch.com/about#%C2%A7how-to-receive-posts-from-epm-sections">how to receive future PM Along AI posts</a>.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p>1. Anthropic. (2025, September 29). <a href="https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents">Effective context engineering for AI agents</a>.</p><p>2. Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan, J., Kikin-Gil, R., &amp; Horvitz, E. (2019). <a href="https://doi.org/10.1145/3290605.3300233">Guidelines for human-AI interaction</a>. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1&#8211;13.</p><p>3. National Institute of Standards and Technology. (2023). <a href="https://doi.org/10.6028/NIST.AI.100-1">Artificial Intelligence Risk Management Framework (AI RMF 1.0)</a>. NIST AI 100-1. (See MEASURE 4.1&#8211;4.3.)</p>]]></content:encoded></item><item><title><![CDATA[004 - How to Calibrate an AI Judge to Match Human Review for Subjective Tasks?]]></title><description><![CDATA[Measure agreement, use disagreements to refine the judging prompt, and test whether the improvement holds.]]></description><link>https://epmresearch.com/p/003-how-to-calibrate-an-ai-judge</link><guid isPermaLink="false">https://epmresearch.com/p/003-how-to-calibrate-an-ai-judge</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Wed, 14 Oct 2026 14:03:11 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!Lvuz!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>An AI judge needs a tested connection to the people whose standards it is meant to apply. We can build that connection by measuring agreement with human reviews, examining disagreements, and revising the judge&#8217;s instructions. Anthropic&#8217;s evaluation guidance recommends closely calibrating model graders with human experts. [1]</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Lvuz!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Lvuz!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png 424w, https://substackcdn.com/image/fetch/$s_!Lvuz!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png 848w, https://substackcdn.com/image/fetch/$s_!Lvuz!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png 1272w, https://substackcdn.com/image/fetch/$s_!Lvuz!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Lvuz!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png" width="662" height="331" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:331,&quot;width&quot;:662,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Reflection and disagreement&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Reflection and disagreement" title="Reflection and disagreement" srcset="https://substackcdn.com/image/fetch/$s_!Lvuz!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png 424w, https://substackcdn.com/image/fetch/$s_!Lvuz!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png 848w, https://substackcdn.com/image/fetch/$s_!Lvuz!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png 1272w, https://substackcdn.com/image/fetch/$s_!Lvuz!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4cb73223-b41a-4eaf-a7ee-44c4aa4e9144_662x331.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>This matters when several outputs could be acceptable. A project risk description, stakeholder response, or engineering explanation can succeed in different ways. The judge needs to recognize the relevant quality standard across those variations. Its fluent explanation of a verdict does not establish that it applies that standard consistently.</p><h3>Measure agreement on the same examples</h3><p>Start with a sample of outputs that humans have already reviewed against a clear standard. Record each verdict and its reason. Include acceptable and unacceptable examples, including difficult cases. Ask the AI judge to assess the same outputs without seeing the human verdicts.</p><p>Compare the decisions case by case. For a simple accept-or-reject review, agreement is the number of matching decisions divided by the number of examples. A spreadsheet is enough to start. The following counts are illustrative.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!7Yy8!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!7Yy8!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png 424w, https://substackcdn.com/image/fetch/$s_!7Yy8!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png 848w, https://substackcdn.com/image/fetch/$s_!7Yy8!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png 1272w, https://substackcdn.com/image/fetch/$s_!7Yy8!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!7Yy8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png" width="1456" height="490" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:490,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:32735,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://epmresearch.com/i/218737577?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!7Yy8!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png 424w, https://substackcdn.com/image/fetch/$s_!7Yy8!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png 848w, https://substackcdn.com/image/fetch/$s_!7Yy8!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png 1272w, https://substackcdn.com/image/fetch/$s_!7Yy8!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9e2e2e03-5189-4bdf-81e9-f9671d8ee641_1462x492.png 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Here, the judge agrees on 28 of 40 examples, or 70%. The total also conceals eight false approvals. Inspect those separately because accepting an inadequate output may have a different consequence from rejecting an acceptable one. A high overall percentage can still leave an important failure pattern unresolved.</p><h3>Turn disagreements into better instructions</h3><p>The useful information is in the reasons for disagreement. Suppose a human rejects a risk entry because it never connects the uncertain event to a project objective. The AI accepts it because it is clearly written. That comparison identifies a missing decision rule in the judge&#8217;s prompt.</p><p>Revise the prompt to make the rule explicit. For example, require the entry to connect an uncertain event to a plausible effect on a stated objective. Preserve acceptable variations in wording. Where the context is insufficient, allow an unclear verdict and route it to human review.</p><p>AI can assist with this revision. Give it the current judging prompt, the human decisions and reasons, and the judge&#8217;s decisions and reasons. Ask it to identify recurring mismatches and propose the smallest changes that address them. Research comparing DSPy optimizers has demonstrated prompt optimization against human labels for hallucination detection. That result supports testing this approach; it does not establish performance for engineering reviews. [2]</p><h3>Repeat the comparison and test the improvement</h3><p>Run the revised judge on the same calibration examples and compare the results with the earlier version. Check whether false approvals declined and whether the revision introduced new disagreements. Keep the judge model and other settings fixed during this comparison so the prompt change is easier to assess.</p><p>Then evaluate a separate sample that was not used to revise the prompt. Keep its human verdicts out of the revision process. This checks whether the new instructions capture a transferable standard. Improvement only on the familiar examples can reflect overfitting.</p><p>Do not expect agreement to rise smoothly with every revision. A change that fixes one mismatch may create another. Retain a revision only when the evidence supports it, and repeat the check when the model, project context, or review standard changes.</p><h3>A prompt for refining the judge</h3><p>Help calibrate an AI judge to our human review standard.</p><p>Inputs: the current judging prompt, our review criteria, and calibration examples with human and AI verdicts and reasons.</p><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;f50f7fe4-4e89-4e42-8922-ed9a13540d3d&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Identify recurring disagreements and the decision rules behind them. Propose the smallest prompt revisions that address those mismatches while preserving valid alternatives.

Flag ambiguous human labels for clarification. Return a concise disagreement summary, a revised judging prompt, and a plan to compare both versions on calibration examples and a separate test sample. Do not claim improvement until the comparisons have been run.</code></pre></div><h2>Notes</h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future. </p><p>Subscribe to EPM Research and learn <a href="https://epmresearch.com/about#%C2%A7how-to-receive-posts-from-epm-sections">how to receive future PM Along AI posts</a>.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p>[1] Grace, M., Hadfield, J., Olivares, R., &amp; De Jonghe, J. (2026, January 9). <a href="https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents">Demystifying evals for AI agents</a>. Anthropic.</p><p>[2] Sarmah, B., Dutta, K., Grigoryan, A., Tiwari, S., Pasquali, S., &amp; Mehta, D. (2024). <a href="https://doi.org/10.48550/arXiv.2412.15298">A comparative study of DSPy teleprompter algorithms for aligning large language models evaluation metrics to human evaluation</a>. arXiv.</p><p>[3] Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., &amp; Stoica, I. (2023). <a href="https://doi.org/10.48550/arXiv.2306.05685">Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena</a>. arXiv.</p>]]></content:encoded></item><item><title><![CDATA[003 - How to Build Self-Improving Iterative Loops for AI-Assisted Workflow?]]></title><description><![CDATA[Connect an approved specification, verification layer, and working environment through small cycles that preserve only confirmed lessons.]]></description><link>https://epmresearch.com/p/003-how-to-build-learning-specification</link><guid isPermaLink="false">https://epmresearch.com/p/003-how-to-build-learning-specification</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Mon, 12 Oct 2026 14:02:20 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!GW7C!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>An approved specification defines what must be true. An approved verification plan defines what evidence will show whether it is true. A loop uses both controls repeatedly to improve successive versions of one work product.</p><p>The principle is simple: <strong>the controls stay fixed; the work product changes</strong>.</p><p>That boundary prevents an agent from making success easier after a weak attempt. If the specification, verifier, sources, tools, permissions, or decision rules need to change, the current run should stop. The agent may propose a change for a future run, but it should not approve its own new rules.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!GW7C!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!GW7C!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png 424w, https://substackcdn.com/image/fetch/$s_!GW7C!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png 848w, https://substackcdn.com/image/fetch/$s_!GW7C!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png 1272w, https://substackcdn.com/image/fetch/$s_!GW7C!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!GW7C!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png" width="800" height="400" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:400,&quot;width&quot;:800,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Title: Alternating specification and verification cycles converging on a reviewable result - Description: Alternating specification and verification cycles converging on a reviewable result&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Title: Alternating specification and verification cycles converging on a reviewable result - Description: Alternating specification and verification cycles converging on a reviewable result" title="Title: Alternating specification and verification cycles converging on a reviewable result - Description: Alternating specification and verification cycles converging on a reviewable result" srcset="https://substackcdn.com/image/fetch/$s_!GW7C!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png 424w, https://substackcdn.com/image/fetch/$s_!GW7C!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png 848w, https://substackcdn.com/image/fetch/$s_!GW7C!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png 1272w, https://substackcdn.com/image/fetch/$s_!GW7C!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa1a4d297-0704-4ed2-b2ae-ff36b5c24075_800x400.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h2>Define the Run Before It Starts</h2><p>A real autonomous loop needs more than &#8220;keep improving.&#8221; It needs a retained baseline, a comparison rule, a budget, and explicit stopping conditions.</p><p>Andrej Karpathy&#8217;s autoresearch project demonstrates the pattern in a tightly controlled setting: an agent modifies one file, trains for a fixed five-minute budget, checks one objective metric, keeps or discards the result, records the experiment, and repeats [1]. The transferable lesson is not the specific code. It is the design of the environment around the agent.</p><p>For professional work, define:</p><ul><li><p><strong>Work product:</strong> the only artifact the agent may change.</p></li><li><p><strong>Fixed controls:</strong> named versions of the specification and verification plan.</p></li><li><p><strong>Environment:</strong> approved sources, tools, runtime, and permissions.</p></li><li><p><strong>Selection rule:</strong> how a passing candidate must improve on the retained version.</p></li><li><p><strong>Limits:</strong> iteration, time, or cost budget.</p></li><li><p><strong>Stops:</strong> success, plateau, unresolved evidence, or a decision reserved for a person.</p></li></ul><p>The baseline is the first retained version. The loop never has to accept a worse candidate merely to keep moving.</p><h2>Separate Findings From Decisions</h2><p>The verifier returns evidence-based findings: MET, NOT MET, or UNRESOLVED. The loop then makes an operational decision:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!xeu2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!xeu2!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png 424w, https://substackcdn.com/image/fetch/$s_!xeu2!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png 848w, https://substackcdn.com/image/fetch/$s_!xeu2!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png 1272w, https://substackcdn.com/image/fetch/$s_!xeu2!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!xeu2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png" width="1456" height="427" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/8f8e150e-0c67-48da-a560-313126551984_1567x460.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:427,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:46921,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://epmresearch.com/i/218736011?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!xeu2!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png 424w, https://substackcdn.com/image/fetch/$s_!xeu2!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png 848w, https://substackcdn.com/image/fetch/$s_!xeu2!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png 1272w, https://substackcdn.com/image/fetch/$s_!xeu2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8f8e150e-0c67-48da-a560-313126551984_1567x460.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>DISCARD means restore the best retained version, not erase the experiment. The loop should preserve a compact lesson about what changed, what evidence appeared, and what should be tried next. Failed attempts can improve the next decision even when they do not improve the work product.</p><p>NASA describes a baseline as an agreed set of requirements, designs, or documents whose changes are controlled [2]. NIST similarly emphasizes documented, repeatable evaluation and a traceable basis for management decisions [3]. Together, these ideas explain why a loop needs fixed controls and a visible record, not merely repeated generation.</p><h2>The Prompt</h2><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;80bc87c6-3733-4fd3-a61d-c3bfec90ebbc&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Run a bounded improvement loop on [WORK PRODUCT].

Fixed controls:
- Specification: [VERSION]
- Verification plan: [VERSION]
- Approved sources, tools, runtime, and permissions: [ENVIRONMENT]
- Selection rule: [HOW A PASSING CANDIDATE IMPROVES ON THE RETAINED VERSION]
- Success, plateau, and stopping conditions: [CONDITIONS]
- Iteration, time, or cost limit: [BUDGET]

Preserve the baseline as the first retained version. Modify only the work product.
Use a separate verifier role or fresh context to apply the approved verification
plan to every candidate.

For each iteration:
1. Choose one change using the retained version and prior lessons.
2. Produce a candidate version.
3. Verify it as MET, NOT MET, or UNRESOLVED with supporting evidence.
4. KEEP it only when all required conditions are MET and it satisfies the
 selection rule. DISCARD it when a condition is NOT MET or it does not improve.
 HALT when a material condition is UNRESOLVED.
5. Record the change, evidence, decision, lesson, and next action. Carry the
 lesson into the next iteration.

Continue without requesting approval until the budget ends or an approved
success or plateau condition is met. Halt if continued work would require
changing a fixed control or making a decision reserved for a person.

Return the best retained version, final verification report, iteration record,
unresolved issues, and proposed control changes for a future run. Do not apply
those proposed changes during this run.</code></pre></div><h2>What Makes the Loop Autonomous</h2><p>The agent does not need a person to approve every attempt. It can choose a bounded change, generate a candidate, invoke the verifier, decide KEEP or DISCARD under the approved rules, record the lesson, and continue for the allowed number of iterations.</p><p>Human authority moves to the boundaries. Before the run, a person approves the objective, controls, environment, budget, and escalation conditions. After the run, a person reviews the best retained version, unresolved issues, and any proposed changes. During the run, human intervention is required only when a material issue is UNRESOLVED or the next action would exceed those boundaries.</p><p>This is what turns a sequence of prompts into an actual loop: state persists across iterations, lessons influence the next attempt, and the best verified result is protected.</p><h2>Questions</h2><h3>Can the loop improve the specification or verification plan?</h3><p>It can propose changes, but it should not apply them during the active run. Approve new control versions first, then begin a new run so every candidate is judged on the same basis.</p><h3>Does DISCARD mean the iteration was wasted?</h3><p>No. The candidate is not retained, but its evidence and lesson remain in the iteration record. That learning should narrow the next change.</p><h3>How many iterations should run without a person?</h3><p>Use the smallest budget that can reveal useful improvement without creating unacceptable cost or risk. High-consequence work needs tighter limits and earlier escalation than reversible drafting work.</p><h2>Notes</h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future. </p><p>Subscribe to EPM Research and learn <a href="https://epmresearch.com/about#%C2%A7how-to-receive-posts-from-epm-sections">how to receive future PM Along AI posts</a>.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p>1. Karpathy, A. (2026). <a href="https://github.com/karpathy/autoresearch">autoresearch</a>. GitHub.</p><p>2. National Aeronautics and Space Administration. (2016). <a href="https://www.nasa.gov/reference/system-engineering-handbook-appendix/">NASA systems engineering handbook: Appendix B&#8212;Glossary</a>.</p><p>3. National Institute of Standards and Technology. (2023). <a href="https://airc.nist.gov/airmf-resources/airmf/5-sec-core/">Artificial Intelligence Risk Management Framework Core: Measure</a>.</p><h2></h2>]]></content:encoded></item><item><title><![CDATA[002 - How to Build a Verification Layer for AI-Assisted Work?]]></title><description><![CDATA[Turn approved acceptance conditions into evidence-based checks, use past examples only for form, and apply the plan in an independent review.]]></description><link>https://epmresearch.com/p/002-how-to-build-a-verification-layer</link><guid isPermaLink="false">https://epmresearch.com/p/002-how-to-build-a-verification-layer</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Fri, 09 Oct 2026 14:03:13 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!YuGy!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>A specification says what must be true. It does not, by itself, say what evidence will be inspected or how a finding will be reached. That missing layer is where confident-looking AI output can escape serious scrutiny.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!YuGy!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!YuGy!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png 424w, https://substackcdn.com/image/fetch/$s_!YuGy!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png 848w, https://substackcdn.com/image/fetch/$s_!YuGy!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png 1272w, https://substackcdn.com/image/fetch/$s_!YuGy!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!YuGy!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png" width="613" height="306" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:306,&quot;width&quot;:613,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:&quot;Title: Front matter option 1 - Description: A sequence of criteria gates leads to an accepted result&quot;,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="Title: Front matter option 1 - Description: A sequence of criteria gates leads to an accepted result" title="Title: Front matter option 1 - Description: A sequence of criteria gates leads to an accepted result" srcset="https://substackcdn.com/image/fetch/$s_!YuGy!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png 424w, https://substackcdn.com/image/fetch/$s_!YuGy!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png 848w, https://substackcdn.com/image/fetch/$s_!YuGy!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png 1272w, https://substackcdn.com/image/fetch/$s_!YuGy!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F54379fd3-e45a-4c3b-8912-3a7ec11b11b1_613x306.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>A verification plan translates each material acceptance condition into an observable check. NASA defines product verification as proof of compliance with specifications, using test, analysis, demonstration, inspection, or a combination of methods [1]. The same principle works for a report, estimate, risk register, or decision brief: define the evidence before the result exists.</p><p>This post stands alone. You can use the method once for a single deliverable, even if no improvement loop follows.</p><h2>Design the Check Before Production</h2><p>For every acceptance condition, specify the evidence, permitted source, method, possible finding, and any judgment reserved for a person. The three findings are deliberately limited:</p><ul><li><p><strong>MET:</strong> the available evidence supports the condition.</p></li><li><p><strong>NOT MET:</strong> the evidence shows the condition failed.</p></li><li><p><strong>UNRESOLVED:</strong> the evidence is insufficient or the decision belongs to a person.</p></li></ul><p>UNRESOLVED is not a softer form of failure. It protects the review from false certainty when sources conflict, calculations cannot be reproduced, or professional judgment is required.</p><p>NIST recommends rigorous testing, uncertainty measures, benchmark comparisons, documented results, and independent review where appropriate. It also notes that independent review can reduce internal bias and conflicts of interest [2]. In practice, that means separating the producing role from the verifying role or, at minimum, starting the verification in a fresh context that does not inherit the producer&#8217;s self-assessment.</p><h2>Use an Example for Form, Not Truth</h2><p>An approved past example can show the expected structure, terminology, and level of detail. Language models can adapt from examples supplied in the prompt [3], which makes a good reference valuable.</p><p>But a reference is not evidence that the new work is correct. The verifier must check the new version against its approved specification and permitted sources. If the example conflicts with them, the conflict should be reported rather than copied.</p><p>The second AI is also not a final authority. Research on LLM judges documents position, verbosity, self-enhancement, and reasoning biases [4]. Independent AI review adds disciplined challenge; it does not replace accountable human approval.</p><h2>Prompt 1: Design the Verification Plan</h2><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;cfbb3fa9-13a8-4729-92fd-a17a9e7950f3&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Design a concise verification plan for the work product defined in the attached
approved specification. Complete this before production begins.

For each material acceptance condition, define:
Condition | Observable evidence | Permitted source | Verification method |
Meaning of MET, NOT MET, and UNRESOLVED | Required human judgment

Use the attached approved example only to match the expected format, structure,
terminology, and level of detail. Do not treat it as evidence for this project.
Flag any conflict between the example, specification, or current sources.

Use UNRESOLVED when the evidence cannot support a finding or when a person must
make the decision. Ask me to correct or approve the plan. After approval, issue
it as a named version and stop before producing the work product.</code></pre></div><h2>Prompt 2: Apply the Approved Plan</h2><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;f4c4e8f6-5c6c-4b5c-a6dc-6a57509cf8dd&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Independently apply the attached approved verification plan to this completed
work product version. Do not revise it and do not rely on the producer&#8217;s
self-assessment.

For each acceptance condition, report:
Evidence inspected | MET, NOT MET, or UNRESOLVED | Finding | Required correction
or human decision

Cite the exact source, calculation, test, or passage supporting each finding.
Do not treat polish or similarity to the format example as evidence of
correctness. Report overall MET only when every required condition is MET, NOT
MET when any required condition is NOT MET, and UNRESOLVED when any material
condition remains unresolved. End with all required human decisions.</code></pre></div><h2>Keep Findings Separate From Actions</h2><p>Verification reports what the evidence supports. It should not quietly rewrite the work product, relax the acceptance conditions, or decide whether a candidate should replace an earlier version.</p><p>For a one-time deliverable, a person can use the report to approve, revise, or reject the work. In a later bounded loop, the same findings can feed explicit KEEP, DISCARD, or HALT decisions. Keeping those decisions outside verification makes the review reusable and easier to audit.</p><p>For higher-consequence work, add requirement identifiers, source owners, calculation files, test environments, reviewer independence rules, and retained evidence. The basic method stays the same: condition, evidence, method, finding.</p><h2>Questions</h2><h3>Must the verifier be a different AI model?</h3><p>No. A different model can add diversity, but a separate role or fresh context is the minimum practical boundary. Critical findings still need the professional review required by the task.</p><h3>Can the verifier fix problems as it finds them?</h3><p>Not during the same review. Mixing correction and verification makes it difficult to know which version was actually checked. Report the finding first; revise and verify a new version separately.</p><h3>What if a condition cannot be checked automatically?</h3><p>Define the available evidence and return UNRESOLVED with the required human decision. A clear escalation is more useful than a fabricated pass.</p><h2>Notes</h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future. </p><p>Subscribe to EPM Research and learn <a href="https://epmresearch.com/about#%C2%A7how-to-receive-posts-from-epm-sections">how to receive future PM Along AI posts</a>.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p>1. National Aeronautics and Space Administration. (2016). <a href="https://www.nasa.gov/reference/system-engineering-handbook-appendix/">NASA systems engineering handbook: Verification and validation definitions</a>.</p><p>2. National Institute of Standards and Technology. (2023). <a href="https://airc.nist.gov/airmf-resources/airmf/5-sec-core/">Artificial Intelligence Risk Management Framework Core: Measure</a>.</p><p>3. Brown, T. B., et al. (2020). <a href="https://arxiv.org/abs/2005.14165">Language models are few-shot learners</a>. Advances in Neural Information Processing Systems, 33, 1877&#8211;1901.</p><p>4. Zheng, L., et al. (2023). <a href="https://arxiv.org/abs/2306.05685">Judging LLM-as-a-judge with MT-Bench and Chatbot Arena</a>. Advances in Neural Information Processing Systems, 36.</p><h2></h2>]]></content:encoded></item><item><title><![CDATA[001 - How Can AI Interview You to Define a Clear Project Spec?]]></title><description><![CDATA[Use a structured interview to identify the work product, surface assumptions, and approve what must be true before production begins.]]></description><link>https://epmresearch.com/p/001-how-can-ai-interview-you-to-define</link><guid isPermaLink="false">https://epmresearch.com/p/001-how-can-ai-interview-you-to-define</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Tue, 06 Oct 2026 14:00:55 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!B4C_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>AI can produce polished material before the task is understood. That speed is useful only after you know what is being produced, who will use it, and what must be true for it to be acceptable.</p><p>The first move is therefore not &#8220;write the report.&#8221; It is &#8220;help me define the work product.&#8221; A work product is the inspectable artifact created by the assignment: a briefing note, risk register, schedule narrative, design option comparison, or another concrete deliverable. It needs a form, an intended user, and a useful outcome.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!B4C_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!B4C_!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg 424w, https://substackcdn.com/image/fetch/$s_!B4C_!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg 848w, https://substackcdn.com/image/fetch/$s_!B4C_!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg 1272w, https://substackcdn.com/image/fetch/$s_!B4C_!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!B4C_!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg" width="1456" height="728" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/2cd006df-d062-4868-b573-90dc26118d08_414x207.svg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:728,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:20212,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/svg+xml&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://epmresearch.com/i/218425556?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!B4C_!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg 424w, https://substackcdn.com/image/fetch/$s_!B4C_!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg 848w, https://substackcdn.com/image/fetch/$s_!B4C_!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg 1272w, https://substackcdn.com/image/fetch/$s_!B4C_!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2cd006df-d062-4868-b573-90dc26118d08_414x207.svg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>A specification records the approved conditions for that work product. NASA&#8217;s systems-engineering guidance offers a useful distinction: a need should describe the problem rather than prematurely prescribe the solution, while a good requirement should be clear, feasible, unambiguous, and verifiable [1]. The same discipline applies to AI-assisted professional work.</p><h2>Let the AI Interview You</h2><p>Most people begin with an incomplete mixture of goals, constraints, preferences, assumptions, and solution ideas. A one-question-at-a-time interview slows the conversation just enough to separate them.</p><p>The interview should establish six things:</p><ul><li><p><strong>Purpose:</strong> What outcome should this work product enable?</p></li><li><p><strong>Users:</strong> Who will read, use, approve, or be affected by it?</p></li><li><p><strong>Form:</strong> What exactly will be delivered?</p></li><li><p><strong>Scope:</strong> What is included, and equally important, what is excluded?</p></li><li><p><strong>Conditions:</strong> What must be true for the result to be acceptable?</p></li><li><p><strong>Authority:</strong> Which assumptions or decisions require a person&#8217;s approval?</p></li></ul><p>This is not limited to software. Government digital-service guidance uses the same logic in a compact form: identify the actor, the need, the goal, and acceptance criteria that describe when the work is done [2]. In a construction or engineering context, the vocabulary may change, but the principle does not.</p><h2>Keep Three Boundaries Clear</h2><p>The specification defines the target. It does not produce the work product, design the verification method, or authorize autonomous iteration.</p><p>That separation matters. If the same conversation quietly moves from defining the need to choosing the solution, important assumptions can disappear into the draft. If it also decides how success will be checked, the criteria may be softened to fit what the AI has already produced.</p><p>The interview should end with a named, approved version. That version becomes a stable reference whether the next step is completed by a person, one AI, a team, or a later improvement loop.</p><h2>The Prompt</h2><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;c23b2c4c-6ea6-4cbf-b57f-f02f4f89c8fc&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Help me define a work product and its specification by interviewing me.

Ask one question at a time. Clarify the problem, who will use the result, the
useful outcome, the smallest worthwhile scope, and the constraints. Bias toward small, 
compartmentalized specs. Make me verify key decisions explicitly so nothing is missed.
Separate needs from proposed solutions and confirmed facts from assumptions. Do not fill
material gaps yourself.

When you understand the task, summarize:
- Goal and intended use
- Work product, form, and intended users
- In scope and out of scope
- Requirements and constraints
- Acceptance conditions describing what must be true
- Assumptions, open decisions, and approval authority

Ask me to correct or approve the summary. After approval, issue the specification
as a named version. Stop there. Do not design the verification method or produce
the work product.

Start by asking me to describe the project idea in my own words.</code></pre></div><h2>What a Good Output Looks Like</h2><p>A strong result is short enough to review and specific enough to govern later work. The work product is named in concrete terms. Scope boundaries are visible. Acceptance conditions describe outcomes rather than vague aspirations. Assumptions are not presented as facts. Open decisions have owners.</p><p>For low-risk work, a concise specification may be enough. For contractual, safety-critical, or multi-party work, add identifiers, source references, tolerances, traceability, and formal change authority. Those are extensions of the same model, not reasons to make every prompt heavy.</p><p>Once the specification is approved, the next question is no longer &#8220;what are we making?&#8221; It is &#8220;what evidence would show that this version meets the approved conditions?&#8221; That is the role of the verification layer.</p><h2>Notes</h2><p>This post is part of the PM Along AI series. <a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">See all episodes here</a>.</p><p>Join the EPM Network to access insights, influence our research, and connect with a community shaping the industry&#8217;s future. </p><p>Subscribe to EPM Research and learn <a href="https://epmresearch.com/about#%C2%A7how-to-receive-posts-from-epm-sections">how to receive future PM Along AI posts</a>.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:&quot;button-wrapper&quot;}" data-component-name="ButtonCreateButton"><a class="button primary button-wrapper" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h2>References</h2><p>1. National Aeronautics and Space Administration. (2016). <a href="https://www.nasa.gov/reference/system-engineering-handbook-appendix/">NASA systems engineering handbook: Appendix C&#8212;How to write a good requirement</a>.</p><p>2. Government Digital Service. (2016). <a href="https://www.gov.uk/service-manual/agile-delivery/writing-user-stories">Writing user stories</a>.</p><h2></h2>]]></content:encoded></item><item><title><![CDATA[000 - How to Build Our Expertise Alongside AI? (PM Along AI Series Index)]]></title><description><![CDATA[Welcome to PM Along AI.]]></description><link>https://epmresearch.com/p/000-how-to-build-our-expertise-alongside</link><guid isPermaLink="false">https://epmresearch.com/p/000-how-to-build-our-expertise-alongside</guid><dc:creator><![CDATA[Pouya Zangeneh]]></dc:creator><pubDate>Fri, 02 Oct 2026 14:03:23 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!Zk0p!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Welcome to <strong>PM Along AI</strong>, a section of EPM Research for engineering students and project professionals.</p><p>AI has become an important part of our research. We have explored practitioners&#8217; expectations about its role in project work [1] and the implications of recent advances for engineering practice [2]. Within our group, we have also been collecting prompts and ideas to test in our own work.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Zk0p!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Zk0p!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png 424w, https://substackcdn.com/image/fetch/$s_!Zk0p!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png 848w, https://substackcdn.com/image/fetch/$s_!Zk0p!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png 1272w, https://substackcdn.com/image/fetch/$s_!Zk0p!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Zk0p!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png" width="1035" height="518" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:518,&quot;width&quot;:1035,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:59183,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://epmresearch.com/i/218424480?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Zk0p!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png 424w, https://substackcdn.com/image/fetch/$s_!Zk0p!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png 848w, https://substackcdn.com/image/fetch/$s_!Zk0p!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png 1272w, https://substackcdn.com/image/fetch/$s_!Zk0p!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa72d5f8f-b52e-4b95-8ed1-ed8d43c266f8_1035x518.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>We increasingly share these ideas with students and professional colleagues. <strong>PM Along AI</strong> gives them a public home where others can find them, try them, and adapt them to their own work.</p><p>A useful prompt does more than request an answer. It helps us define the task, examine the available inputs, make assumptions visible, and describe how a good result will be checked. Documenting why we use a prompt also helps others decide where it fits their work.</p><h2>Series Index</h2><p>Here is the list of articles in the PM Along AI series. This list will be updated as new episodes are published.</p><p><strong><a href="https://epmresearch.com/p/000-how-to-build-our-expertise-alongside">000 - How to Build Our Expertise Alongside AI?</a> </strong>Why this series exists and how to follow it.</p><p><strong><a href="https://epmresearch.com/p/001-how-can-ai-interview-you-to-define">001 - How Can AI Interview You to Define a Clear Project Spec?</a> </strong>Use a structured interview to surface assumptions and approve what must be true before production begins.</p><p><strong><a href="https://epmresearch.com/p/002-how-to-build-a-verification-layer">002 - How to Build a Verification Layer for AI-Assisted Work?</a> </strong>Turn acceptance conditions into evidence-based checks and apply them in an independent review.</p><p><strong><a href="https://epmresearch.com/p/003-how-to-build-learning-specification">003 - How to Build Self-Improving Iterative Loops for AI-Assisted Workflow?</a> </strong>Link spec, verification and working environment through small cycles that keep only confirmed lessons.</p><p><strong><a href="https://epmresearch.com/p/003-how-to-calibrate-an-ai-judge">004 - How to Calibrate an AI Judge to Match Human Review for Subjective Tasks</a> </strong>Measure agreement with human reviewers, refine the judging prompt, and test that the improvement holds.</p><p><strong><a href="https://epmresearch.com/p/005-ask-ai-what-human-input-could">005 - Ask AI What Human Input Could Improve Its Work?</a> </strong>Find the smallest useful human contribution and verify its effect on the next version.</p><p><strong><a href="https://epmresearch.com/p/006-how-to-build-a-simple-ai-scheduled">006 - How to Build a Simple AI Scheduled Task of Review and Sending Emails with Improvable Structure?</a> </strong>Keep the procedure in editable files and let a weekly schedule run each step separately.</p><p><strong><a href="https://epmresearch.com/p/009-the-case-for-hybrid-local-and">007 - The Case for Hybrid Local and Frontier AI: Sensitive Project Data.</a> </strong>Keep sensitive project files on a machine you control, send frontier models only cleaned questions, and use that private space to experiment.</p><h2>Prompting Idea</h2><h3>Map your work before deciding what to learn</h3><p>Growing alongside AI means developing the expertise to direct its contribution and judge its results. That starts with understanding the work itself.</p><p>AI will not affect every task in the same way. A scheduler may use it to organize inputs or test construction logic. A risk analyst may use it to identify assumptions or compare scenarios. In both cases, the person still needs enough domain knowledge to notice missing evidence, weak reasoning, and project conditions that the AI does not understand.</p><p>Our work on project risk has highlighted the need to teach young engineers to challenge AI outputs [3]. Verification should be part of learning rather than a final administrative check. When we test an AI answer against evidence and professional knowledge, we also deepen our understanding of the task.</p><p>The following two prompts use that principle. The first creates a confirmed map of your work. The second uses that map to choose expertise and a practical experiment.</p><h4>Prompt 1: understand my work</h4><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;6870f35d-6b27-4460-a408-4f6041a48480&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Interview me to understand how my work is done and where AI might help.

Ask one question at a time. Focus on the purpose of each task, how work
flows from inputs to outputs, where human judgment matters, how quality is
checked, and what repeatedly causes difficulty. Separate facts from my
assumptions.

Summarize what you learn as:
Task | Workflow | Human judgment and quality checks | Current difficulty

Ask me to correct the map before suggesting any use of AI.</code></pre></div><h4>Prompt 2: decide what expertise to build</h4><div class="highlighted_code_block" data-attrs="{&quot;language&quot;:&quot;plaintext&quot;,&quot;nodeId&quot;:&quot;b88004bb-af24-42f4-92f1-ad2a38c07e19&quot;}" data-component-name="HighlightedCodeBlockToDOM"><pre class="shiki"><code class="language-plaintext">Using my confirmed workflow map, identify where AI can assist now, what may
plausibly change over the next few years, and what human expertise should
grow alongside it.

Separate demonstrated capabilities from predictions and state the main
uncertainties. Recommend three learning priorities and one small experiment
I can try this week. For each recommendation, explain how I should verify
that it improves the work rather than merely making it faster.</code></pre></div><h3>Turn the result into practice</h3><p>Choose one recurring task rather than trying to redesign an entire role. Correct the workflow map before asking for recommendations because a mistaken map will produce the wrong learning priorities.</p><p>Treat capability forecasts as scenarios, not facts. Tools, costs, organizational policies, and regulatory expectations can change. The useful question is which expertise remains valuable across several plausible futures.</p><p>Then run the proposed experiment on a low-consequence task. Compare the result with your current method. Check the quality, time required, errors, assumptions, and effort needed for review. Keep the practice only if it improves the work without weakening accountability.</p><p>This approach turns AI adoption into deliberate learning. The aim is not to predict every capability. It is to build enough understanding to make better choices as the technology and the work develop.</p><h2>Notes</h2><p><strong>Join the EPM Network</strong> to access insights, influence our research, and connect with a community shaping the industry&#8217;s future.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://epmresearch.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://epmresearch.com/subscribe?"><span>Subscribe now</span></a></p><p>Support us by sharing this article with your friends and colleagues, or over social media.</p><h3>Receive PM Along AI</h3><p>To receive these posts, click <strong>Unsubscribe</strong> at the bottom of an email you have received from <strong>EPM Research</strong>. On the subscription preferences page, toggle <strong>PM Along AI</strong> on and save your preferences if prompted.</p><h2>References</h2><p>1. Zangeneh, P., &amp; Ghorab, K. (2025, March 12). <a href="https://epmresearch.com/p/transformative-technologies-in-construction">Transformative technologies in construction and project management: 2025 outlook and survey results</a>. EPM Research.</p><p>2. Zangeneh, P. (2026, September 11). <a href="https://epmresearch.com/p/ai-takes-shape">AI takes shape: What recent AI advances mean for architecture, engineering, and construction</a>. EPM Research.</p><p>3. Zangeneh, P. (2025, November 27). <a href="https://epmresearch.com/p/brace-for-ai-risk-management-failures">Industry should brace for the first wave of AI-driven risk management failures</a>. EPM Research.</p>]]></content:encoded></item></channel></rss>