Research,
made intelligible.

Research is not synonymous with searching, reading, collecting citations, writing a paper, or producing something novel. Those can be parts of research. The load-bearing operation is the connection between a question, a method, evidence or argument, and a conclusion whose limits remain visible. The exact meaning of research varies across disciplines: an archive, an interview, an experiment, a mathematical proof and a designed artifact can each carry inquiry, but not the same kind of warrant.

A useful working loop is question → investigation → interpretation → criticism → revision. This is a teaching model, not a claim that every project follows five tidy stages. You may discover that your question is ill-posed, your measure misses the phenomenon, or an older result already answers it. Revising the question can be progress, not failure. Philosophical accounts of scientific method examine this diversity rather than giving every field one mechanical recipe. [S01]

Learning, investigation and contribution

You can do careful research to learn what is already known. An original research contribution additionally changes what others can know, explain, measure or do. A literature synthesis can contribute by resolving a disagreement or exposing a boundary; a replication can strengthen or challenge confidence without inventing a new mechanism. Personal learning and public novelty are therefore distinct achievements.

Do not begin by promising certainty. Begin by identifying what kind of uncertainty you can responsibly reduce. A defensible negative result, a counterexample, a narrower claim, an improved measurement or a well-supported account of why evidence is insufficient can all be useful outputs.

The four questions underneath the work

1. What exactly am I trying to find out?
2. What observations, sources or arguments would bear on it?
3. Why does my method connect those materials to an answer?
4. What would make the answer weaker, wrong or inapplicable?

Exclusive means restricted to some people or things, or excluding others. It is not a synonym for exhaustive. “Not an exhaustive literature or prior-art review” is a coverage disclosure: selected sources were used, but the surrounding work has not been mapped comprehensively enough to make unrestricted priority claims.

Make the denominator visible

“We searched the literature” hides the search universe. A useful statement names the topic, eligible source types, databases or collections, languages, date range, last search date, and important access limitations. Breadth has a cost: more material to screen, reconcile and keep current. Narrowness has a cost too: excluded contexts can contain the closest precedent or strongest counterexample. [S04]

A sound stopping rule is not “I feel finished.” It might be completion of a prespecified search across selected sources plus citation chasing and assessment of known gaps. Iterative theoretical or qualitative work can use different stopping rationales, but must explain them. Budget exhaustion is a legitimate constraint when disclosed; it is not evidence of completeness.

A topic names an area. A problem names something not adequately understood or served. A question states what you want to learn. An objective states what you will accomplish. A hypothesis offers a candidate answer that can meet criticism. A contribution is what the completed work adds—not merely what you intend to build.

Choose the kind of answer

• Descriptive: what exists, how often, or in what forms?
• Comparative: how do groups, cases, methods or interpretations differ?
• Predictive: how well can we anticipate an outcome on genuinely new cases?
• Causal: what would change under a specified intervention?
• Interpretive: what meanings or practices organize experience in context?
• Formal: what follows from these definitions and assumptions?
• Constructive/design: what artifact can meet these requirements, and how will that be evaluated?
• Normative: what ought to be done, under which defensible values and reasons?

These categories can interact, but their evidence obligations differ. Prediction does not automatically identify causes. Interviews do not estimate population prevalence without an appropriate sampling/inference design. A prototype demonstrates that something can be built, not that it improves outcomes. Data can inform a normative judgment without supplying its values by themselves.

Give the question edges

Name the population or setting, object of study, comparison where relevant, outcome or concept, timescale, and unit of analysis. Frameworks such as PICO can help with some intervention questions; they are aids, not universal molds for mathematics, history or interpretive work. Ask whether access, time, expertise and ethical permissions make the question feasible.

Start with one main question and a small number of dependent questions. If each subquestion demands a different dataset, population and disciplinary method, you may have several projects rather than one ambitious paper.

Ontology concerns what kinds of things exist or are treated as real in an inquiry. Epistemology concerns what counts as knowledge and how it can be justified. Methodology is the rationale and logic connecting questions to methods. Methods are the actual procedures: interviewing, sampling, coding, proving, modeling or testing. Axiology concerns the role of values. These are not decorative opening-section words; they matter when they change what evidence you seek or which conclusion you can draw. [S01]

For example, treating “understanding” as a benchmark score is not merely choosing a convenient metric. It is adopting an operational representation of a contested concept. You may justify that representation for a narrow task without claiming it exhausts the phenomenon. Ask what it reveals, what it discards and what would invalidate the mapping.

Three useful reasoning movements

• Deduction: deriving what must follow if stated premises hold. Soundness needs valid inference and appropriate premises.
• Induction: generalizing beyond observed cases. Its warrant depends on how the cases and world support that extension.
• Abduction: developing a plausible explanation for observations. The best current explanation remains open to rivals and new evidence.

Exploration generates possibilities. Confirmation tests specified expectations. Neither is inherently superior; hiding an exploratory choice as if it were a prior prediction is the problem. Preregistration can make that distinction more visible, but it does not make a weak theory or measurement strong. [S05]

Representation is a research decision

A transcript makes chronology easy to recover. A claim–source table makes support easy to inspect. A causal model makes intervention assumptions explicit. A proof-dependency graph makes lemmas visible. None is interchangeable with the others. In particular, a workflow arrow or dependency edge is not automatically a causal relationship.

PRISMA is a reporting guideline: it helps readers see why a review was done, what was done and what was found. A completed checklist or flow diagram does not certify excellent conduct, absence of bias or a true conclusion. PRISMA-ScR is the related extension for scoping reviews; the extension should not be confused with the main reporting guideline. Choose the relevant guidance for the actual review type. [S02] [S03]

Do not label a handful of web searches a systematic review because the final article has a PRISMA-style diagram. Conversely, a carefully scoped narrative synthesis can be valuable without that label. Review work has its own research question and method; it is not the preliminary reading stage stretched into a paper.

Conduct guidance and reporting guidance are complementary. The Cochrane Handbook provides detailed methods for intervention reviews. Its procedures are useful evidence for that domain, not a command to force every historical or theoretical inquiry into a clinical-review template. [S04]

In academic and architectural work, relevant prior work may be a paper, technical report, specification, documented system, implementation or earlier conceptual argument. The newest paper is not necessarily the closest precedent. You are looking for work that shares the problem, mechanism, assumptions, operating conditions or contribution, including work described using different vocabulary.

“Prior art” also has specific legal meanings in patent practice. This handbook uses it in the broader research-comparison sense; it does not perform a patent search, determine patentability or offer legal advice. Academic originality and a legal novelty determination are not interchangeable.

Four questions that sharpen the comparison

1. What is the closest existing approach—not merely a convenient weak baseline?
2. Under which assumptions and conditions does it work?
3. What precisely changes in our approach: mechanism, guarantee, integration, representation, evidence or application?
4. What supports that difference, and why does it matter?

A gap is not yet a contribution. “Nobody studied this exact combination” may reflect lack of importance, an obvious extension, inaccessible data or different terminology. Explain why filling the gap improves understanding or practice. A contribution can be a carefully evaluated combination, a boundary result, a negative result, replication or improved measurement; it need not claim every component is unprecedented.

A safer novelty sentence

Instead of “This is the first system to make research trustworthy,” write a bounded comparison: “Unlike the selected approaches A and B, this proposal explicitly separates artifact-version review from action authorization; whether that separation improves review outcomes remains to be evaluated.” This is a synthetic phrasing example, not a finding about any named system.

Finding no predecessor in a limited search supports “we did not identify one within this search,” not “none exists.” Record searched terms, sources, dates, exclusions and unresolved neighbors before making priority claims.

Break the question into a few concept blocks. For each, list synonyms, older terms, acronyms, spelling variants and broader/narrower concepts. Combine alternatives within a block using OR; combine different concepts using AND where the database supports it. Phrase searching, truncation, proximity syntax, fields and controlled vocabulary differ across systems. Test syntax rather than transplanting a query blindly. [S04]

("human review" OR "human oversight")
AND ("version history" OR provenance OR "artifact version")
AND ("research assistant" OR "AI agent")

This is an illustrative query, not a search executed for an exhaustive architecture review. An AND-heavy query can miss papers that discuss only part of your mechanism. Run focused subqueries as well as combination queries. Use known relevant seed documents to check whether the strategy can retrieve material it ought to find.

Broaden the source channels deliberately

Use appropriate scholarly indexes and disciplinary databases, library catalogues, proceedings, preprint servers, registries, standards and relevant technical repositories. General web search is useful for discovery, but rankings are not a sampling frame. Backward citation chasing follows a paper's references; forward chasing finds later work that cites it. Both can reveal vocabulary and debates, but can also overrepresent well-connected schools.

Grey literature includes material outside conventional commercial academic publishing, such as reports, theses and some working papers. It can reduce publication blind spots while creating different appraisal and discovery challenges. Preprints provide early access but should be labeled with version and review status.

Keep a search log

Record the exact query, platform, fields/filters, date, coverage limits, returned/exported counts when actually observed, and what changed between searches. A DOI or metadata match identifies a record; it does not mean you read the article, evaluated its method or reproduced its result.

Eligibility criteria say which questions, populations, contexts, methods, outcomes, dates or source types qualify. Inclusion and exclusion rules should serve the question, not the direction of the results. Pilot them on a small set to find ambiguity. If they change during the review, record why, when and which earlier decisions need revisiting. Systematic-review practice makes the search and selection trail explicit. [S02] [S04]

Distinguish a record found in a database, a report or publication, and the underlying study. One study may produce multiple reports; the same report may appear in several databases. Deduplicating citations does not automatically prevent double-counting participants or evidence. Corrections, translations, preprints and final publications also need version-aware handling.

Screen in stages, preserve reasons

Title/abstract screening is triage. Full-text assessment decides eligibility when the abstract is insufficient. “Full text unavailable” differs from “ineligible after reading.” Track retrieval failures rather than silently assigning exclusion reasons. Use relevant reporting standards to show what entered, left and remained in the process; do not invent flow counts after the fact.

Independent screening and extraction can reduce particular reviewer errors, but require a clear disagreement-resolution process and an honest description of independence. Two agents sharing the same prompt, context and model may repeat the same mistake. Neither multiple reviewers nor a software-assisted workflow guarantees unbiased selection.

Two threats worth naming

Selection bias can arise when the included evidence systematically differs from the relevant universe. Publication bias arises when availability is related to results or their perceived interest. Language, database and access restrictions may introduce additional asymmetries. A useful limitations section explains how those mechanisms might change your conclusion—not simply that “more research is needed.”

Read at different resolutions. First orient: question, contribution, study type or argument structure, and relevance. Then examine methods, definitions and evidence for the claims you intend to use. Finally reconstruct the critical steps and compare them with alternatives. You need not read every reference cover to cover, but must not imply whole-source reading after inspecting only a title, abstract or excerpt.

Keep four voices separate

• Quotation: the source's exact words, with an exact location.
• Paraphrase: your faithful restatement, still attributed.
• Interpretation: what you think the source means or implies.
• Synthesis: a new connection or conclusion you draw across sources.

Paraphrasing is not deleting the citation. A sentence can be original in wording and still borrow its idea or evidence. Check qualifications: “may,” “under these assumptions,” “in this sample,” “pointwise,” “on average” and “within the model” are often the load-bearing part of a result.

A useful extraction record

Save identity/version, inspected pages or sections, the consequential claim, design/data or proof structure, assumptions, uncertainty, limitations, competing interpretations and the precise role the source plays in your work. For quantitative studies, extract the outcome definition, estimate, uncertainty, population, comparison and relevant design details together. For qualitative work, preserve context, participant perspective and the path from material to interpretation. For mathematics, preserve quantifiers and dependencies.

A claim ledger is not a truth machine. A hash identifies bytes; a citation resolves a document; a reviewer may still misunderstand either. The ledger helps locate the next correction. When a claim changes, revisit the dependent prose and checks instead of merely editing a bibliography entry.

Quantitative approaches represent aspects of a phenomenon numerically and use mathematical/statistical reasoning. Qualitative approaches investigate meaning, experience, practice, process and context through appropriate interpretive procedures. Mixed methods intentionally integrates quantitative and qualitative components so their relationship answers a question neither handles adequately alone. Adding one quotation to a survey does not, by itself, establish a mixed-methods design.

Experiments manipulate conditions; observational studies do not assign the exposure of interest in the same way. Surveys can measure reported attitudes or behavior but inherit question, sampling and response problems. Case studies examine bounded cases in depth; ethnographic approaches examine practices in social contexts. Historical work critically interprets records, provenance, absence and context. Formal work uses definitions and argument; design research constructs and evaluates artifacts. These are families, not mutually exclusive boxes or a universal quality ranking. [S14]

Units and comparison matter

Separate the unit of observation from the unit of analysis. Thousands of events from ten people do not automatically create thousands of independent participants. Repeated observations, clustered organizations and temporal dependence affect inference. The sample is what you actually observe; the target population is what you want to say something about.

Probability sampling supports particular population inferences under its conditions. Purposive sampling selects informative cases for a stated inquiry. Convenience sampling uses what is available. None should be described as representative simply because the sample is large. Qualitative transferability also needs contextual reasoning, not an automatic claim that a theme holds everywhere.

Plan evaluation before collecting convenient evidence

Specify the comparison, measurement, analysis, exclusions and important failure modes. Use pilot work to test feasibility and refine procedures, clearly separating it from a confirmatory test of a prespecified claim. Reporting guidelines can help reveal missing decisions; they do not substitute for domain expertise or ethical approval. [S05] [S08]

A construct is the concept of interest: trust, learning, fatigue, usefulness. An operationalization specifies how that concept enters observation or analysis. A variable can take different values; a measure gives the procedure for obtaining them. Ask whether the measure captures the intended construct or merely a convenient proxy.

Reliability concerns consistency under specified conditions. Validity concerns whether the interpretation or inference is justified. Construct validity concerns the measure–concept connection; internal validity concerns the credibility of an inference within a study, often causal; external validity concerns extension beyond studied conditions. Terminology and validation practices differ across disciplines. A reliability coefficient is not a universal certificate of validity. [S15]

Data do not arrive unmediated

Collection instruments, prompts, logging decisions, coding rules, units and exclusions shape the dataset. Keep a data dictionary, identify how each field was produced, and preserve transformations. Separate raw observations from cleaned data and derived features. Record software versions, data versions and decisions about outliers or impossible values. Sensitive raw data may need restricted retention, not indiscriminate preservation.

Missingness has mechanisms. MCAR, MAR and MNAR describe different assumptions about how missing data relate to observed and unobserved values. They are not labels a tool can reliably infer from a blank cell. Dropping incomplete cases or replacing values with a mean can change the target population, variability and bias. Choose and justify a method; use sensitivity analysis where assumptions cannot be settled from observed data alone.

Protect the evaluation boundary

Training, validation and test sets have different jobs. Repeatedly tuning to a held-out test set makes it part of development. Leakage occurs when information unavailable at the intended prediction time enters training or assessment. A high benchmark score is less informative if examples, labels or task patterns leaked across that boundary.

Association describes how variables vary together. Prediction uses available information to anticipate outcomes. Causal inference asks what would happen under a specified change, compared with an alternative. A counterfactual asks about an alternative outcome under a different condition for the relevant unit or setting. Moving between these questions requires information and assumptions, not merely a more confident verb. [S16]

A confounder can influence both the exposure/treatment and outcome, creating a competing explanation for their association. A mediator lies along a proposed causal pathway. A collider is a common effect of variables; conditioning on one can create bias. These are roles in a model, not permanent properties of variable names. Adding every available variable to a regression is not a universally safe adjustment strategy.

What would identify the effect?

An estimand is the precise target quantity or contrast. Identification asks whether it can be expressed in terms of the available information under stated assumptions. Estimation computes an estimate using data. A beautifully estimated association is not an identified causal effect simply because software reports a narrow interval.

Random assignment can support causal comparison by breaking systematic treatment assignment links, but it does not automatically solve attrition, noncompliance, interference, measurement problems or generalization. Observational causal inference is possible under appropriate designs and assumptions; it is not forbidden, nor does it follow from association alone. Common concerns include exchangeability, positivity/overlap, consistent treatment definitions, and whether one unit's treatment affects another's outcome.

Diagrams are commitments

A causal graph makes assumed relationships inspectable. It does not prove those relationships from the act of drawing arrows. An execution-order graph, citation network or dependency map is not automatically a causal model. State which kind of edge you mean, and examine which alternative models could explain the same observations.

An effect size describes magnitude in a chosen scale. A standard deviation describes spread among observations; a standard error describes uncertainty of an estimator under a model/design. A large dataset can make a tiny effect statistically detectable without making it important. Conversely, an important possible effect can remain imprecisely estimated.

The p-value, carefully

A p-value assesses how incompatible the observed result is with a specified null model, using a defined test statistic and extremeness rule. More precisely, under that model and its assumptions, it is the probability of obtaining a statistic at least as extreme as the observed one. It is not the probability that the null hypothesis is true, that your theory is false, or that “chance caused the result.” The ASA specifically warns against treating a threshold as a substitute for scientific reasoning. [S06]

Illustration only: p = .03 does not mean “97% probability our mechanism is correct.” The model, design, multiplicity, effect magnitude, uncertainty and competing explanations still matter. A non-significant result is not automatically evidence of no effect. An equivalence claim needs an appropriate design, meaningful bounds and analysis—not merely failure to reject a null.

Intervals do different jobs

A frequentist 95% confidence interval comes from a procedure with a specified long-run coverage property under its assumptions. It is not, in that framework, a 95% probability statement about the fixed parameter lying in this particular realized interval. A Bayesian credible interval summarizes posterior probability conditional on the model, data and prior. A prediction interval concerns future observations rather than just an unknown mean or parameter. None automatically captures every source of bias or model misspecification. [S07]

Design for honest uncertainty

Power is the probability of rejecting the null under a specified alternative and design. Sample-size planning depends on the question, effect of interest, noise, precision, clustering and error tradeoffs; there is no universal “30 participants is enough.” Running many tests or trying many pipelines changes error risks. Prespecify relevant decisions, report multiplicity/deviations, distinguish exploration, and use suitable corrections or validation—not selective reporting of the smallest p-value. [S05] [S06]

Bayesian updating combines prior information with a likelihood to obtain a posterior. It is not a shortcut around assumptions: priors, model choices and sensitivity still need justification. Frequentist and Bayesian quantities should not be blended into one vague confidence number.

Qualitative inquiry is not simply quantitative inquiry with too few participants. It may investigate how people make sense of experiences, how practices develop, or how institutions and contexts shape action. Interviews, observations, documents and other materials are generated and interpreted within a chosen methodological approach. Clear reporting makes that approach and its relationship to the findings inspectable. [S08]

Coding assigns analytic labels to material. A theme is not necessarily the most frequently used word or the most frequently mentioned topic; it can represent a patterned meaning relevant to the question. Whether coding is inductive, deductive or combined, explain the path from material to interpretation and what role theoretical commitments played.

Rigor without pretending the researcher disappears

Reflexivity examines how the researcher's position, assumptions, relationships and decisions shape the work. It is not a ritual confession that cancels bias. An audit trail records analytic development. Attention to negative or deviant cases can reveal where an emerging interpretation is too broad. Contextual description helps readers judge transferability.

Terms such as credibility, dependability, confirmability and transferability are used in some traditions to discuss trustworthiness. Do not turn them into a universal checklist detached from the epistemology of the study. Inter-coder agreement can be appropriate for some coding designs, while other approaches do not treat agreement between coders as their central quality criterion. Justify the standard rather than claiming one method settles every tradition.

Sampling and saturation need a rationale

Purposive or theoretical sampling may seek cases that illuminate a concept rather than approximate a population distribution. Saturation has multiple meanings and is not a magic sample-size threshold. Explain what was expected to become sufficient—codes, conceptual development, variation or something else—and whether saturation is appropriate to the chosen approach at all.

Quotations should connect an interpretation to material while protecting participants. A vivid quote does not prove prevalence, represent every participant, or absolve you from showing analytic reasoning. Anonymity also depends on context: removing a name may not prevent re-identification.

Critical appraisal evaluates how a source's design, conduct, analysis and reporting affect the inferences you want to use. Risk of bias concerns systematic error, not merely whether a paper is well written or published somewhere familiar. Certainty of evidence concerns confidence in a body of evidence for a particular conclusion, not a permanent score assigned to an author.

Separate study limitations, reporting omissions, imprecision, inconsistent findings and applicability. Peer review is a useful scrutiny mechanism, not a guarantee. Citation counts measure influence and attention, not truth. A preregistration, open dataset or available code improves inspectability but does not, by itself, establish a valid method or result.

Compare before combining

Build a synthesis matrix: question, population/context, design, construct/outcome, result or argument, uncertainty, assumptions, limitations and relevance. Several papers may rely on the same dataset or study; that is not independent corroboration. Differences in measures, populations, follow-up periods or implementation can make nominally similar outcomes non-equivalent.

Meta-analysis is appropriate only when combining estimates makes substantive and statistical sense. Study weights are not simple votes. Heterogeneity may reflect sampling variation, genuine effect variation, design differences or bias; no single statistic resolves all of those. Publication and selective-reporting processes can distort a pooled estimate. A precise summary of biased inputs remains a problem.

Write the relationship, not the pile

A synthesis might show convergence under certain assumptions, disagreement explained by context, an unresolved conflict, a measurement mismatch or a missing comparison. “A says X; B says Y; C says Z” is organization, not yet synthesis. Add the supported relationship and its boundary.

Use domain-appropriate appraisal tools rather than importing one universal evidence ladder. The strongest design for a treatment-effect question is not automatically the strongest design for understanding lived meaning, establishing a theorem, interpreting an archive or evaluating a system's failure behavior. [S04] [S08] [S11]

A theoretical framework organizes concepts and proposes explanatory relationships. A model is a selective representation for a purpose. A mathematical theorem is a precise statement supported by a proof under stated assumptions. A conjecture is a proposition not yet established in the relevant sense. These should not be collapsed into the single phrase “we have a theory.”

For mathematical work, expose definitions, domains, quantifiers, assumptions and necessary lemmas. “For each x there exists a y” is generally not the same as “there exists one y for every x.” Pointwise behavior is not automatically uniform behavior. A finite numerical check does not establish a universal statement over an infinite domain. Failure to find a proof is not a disproof; failure of one attempted proof is not necessarily failure of the theorem.

Formal checking has a scope

A formal proof assistant can check a formal proposition relative to its logic, axioms, definitions, dependencies and trusted checking process. The proposition may still differ from what the prose claims. Inspect the intended statement, actual target, solution and transitive assumptions. Lean's guidance explicitly separates checking mechanisms and remaining trust conditions; an incomplete proof can be visible through sorryAx. [S12]

Do not infer proof correctness from a successful LaTeX build, available source file or badge. Nor should all formal checks be dismissed as mere compilation: correctly scoped kernel/checker validation is substantive evidence about the formal artifact. The point is to state exactly what was checked, what was trusted, and whether the formal statement corresponds to the intended claim.

Novelty and explanation remain separate

A valid proof of a known result is not automatically new research. An original result still needs a correct argument. An elegant explanatory framework must face alternatives and constraints rather than merely re-labeling observations. Empirical theories may produce testable implications; pure mathematical claims face proof and counterexample obligations rather than a forced experimental template.

Architecture and design-oriented research can contribute an artifact, mechanism, representation, contract, design principle, evaluation or explanatory account. Its value depends on the problem and the evidence supporting the contribution. A feature list alone does not establish research novelty; a working prototype alone does not establish user benefit.

Separate four layers of evidence

1. Proposal: requirements, mechanism, alternatives, assumptions and tradeoffs are described.
2. Implementation: particular behavior exists in a particular artifact/version.
3. Verification: scoped properties or requirements were checked by named procedures.
4. Evaluation: evidence addresses usefulness or effects in specified contexts, with suitable comparisons.

These are not a universal maturity ladder. A well-argued proposal may be a legitimate paper type, and evaluation can shape design long before a final implementation. The distinction prevents a polished diagram from being treated as tested operation. ACM similarly distinguishes artifact availability/evaluation from results validation by another person or team. [S11]

Make the mechanism and failure boundary visible

State who acts, what data and authority each component has, what must remain invariant, and what happens when a dependency is unavailable, stale or interrupted. Separate information flow from permission to act. If you claim recovery, specify the failure model: process interruption, network failure and power loss are not the same test.

Use realistic baselines, negative cases and the actual supported environment. Measure the total burden, not just the successful substep: preparation, reading, review, correction, integration and recovery can erase an apparent speed gain. Human evaluation needs appropriate consent and careful task design; a developer's self-review is not independent acceptance.

An illustrative evaluation question is: “Does version-aware evidence review reduce acceptance of stale claims compared with ordinary notes, after review overhead is included?” It still needs an operational outcome, protocol, comparison and real observations. This handbook supplies no measured result for it.

Terminology varies. In some communities, reproducibility emphasizes obtaining consistent computational results using the same data/code, while replication emphasizes new observations or independent implementations addressing the question. ACM's badge policy distinguishes results reproduced using author-provided artifacts from results replicated without them, by another person/team. State the operation rather than assuming every reader shares one vocabulary. [S11]

Preserve versioned protocols, data dictionaries, code/environment information, parameters, seeds where relevant, source identities and meaningful output checks. A random seed does not control every source of nondeterminism. A container does not prove a correct analysis. A successful rerun can reproduce a bug. Artifact availability is not the same as independent verification of the main result.

Provenance is the history of a claim or artifact

Record where it came from, which transformations created it, which version a check covered, and who or what performed that check. A digest helps identify exact bytes, not scientific truth. A statement can remain byte-identical while its support changes because an important source is corrected or a necessary review fails. Keep current support and historical receipts distinct.

The FAIR principles concern findability, accessibility, interoperability and reusability. They include identifiers, metadata, qualified references, provenance and clear reuse conditions. Accessible does not mean that every file must be public: authentication and authorization are permitted where necessary. Sensitive data can need controlled access; openness does not override privacy or consent. [S10]

Corrections are part of the system

Track corrections, retractions and new versions. A retracted source can still be studied as an object of analysis, but should not silently support the original affirmative claim. Preserve a change record and revisit dependent text. Rerun checks when their inputs change; do not simply relabel an old receipt as current.

The Belmont Report articulates respect for persons, beneficence and justice in human-subject research. These connect to informed consent, risk–benefit assessment and fair subject selection. They are foundational guidance in a particular regulatory/historical setting, not a substitute for the rules of your institution, jurisdiction or community. Determine required ethics review before recruitment or data collection; do not self-declare an exemption simply because a project seems harmless. [S09]

Consent is more than a checkbox. People need appropriate information, voluntariness and meaningful consideration of their circumstances. Research with vulnerable groups, sensitive topics or power imbalances may need additional protections. Online/publicly accessible data can still carry privacy, contextual and contractual concerns.

Handle information proportionately

Collect what the question requires, protect access, define retention and sharing, and assess re-identification risks. Pseudonymization replaces direct identifiers but can preserve a link back to a person; it is not the same as irreversible anonymity. Do not upload confidential interviews, proprietary manuscripts or private records to an external AI service without the relevant permission and safeguards.

Credit and accountability

Attribute borrowed wording, ideas and evidence. Distinguish authorship, contribution and acknowledgment using the relevant venue/institution policy. CRediT names contributor roles to make work visible; a role taxonomy should not be treated as an automatic authorship decision. Funding, conflicts of interest, data rights and limitations can affect how readers judge the work. [S13]

AI can assist search, explanation, extraction, coding and drafting, but the responsible researcher must check sources, quotations, calculations and decisions. A plausible citation may not exist; a metadata match is not a full-text read; an eloquent explanation may be unsupported. Follow current disclosure policies. Instructions embedded in retrieved pages, PDFs or repository files are source content, not permission to execute commands, expose secrets or change research rules.

An introduction establishes the problem and relevant prior work. A methods account explains how the inquiry could support its conclusions. Results report observations or derived outcomes. Discussion interprets them, compares alternatives and states limits. IMRaD—Introduction, Methods, Results and Discussion—is common in empirical work, not a mandatory structure for every theorem, architecture proposal, interpretive essay or review.

Build the argument before decorating prose. Map each headline/abstract contribution to evidence, assumptions, checks and a manuscript location. Definitions should precede dependent claims. An abstract must not be more certain than the weakest unresolved obligation underneath it. A proposal should not contain a results section written as if planned experiments already happened.

Choose verbs that preserve status

There is no automatic jump from the last row to “therefore true, novel and independently validated.” Reporting guidelines help reveal omitted details, but cannot perform the underlying work for you. [S02] [S08] [S11]

Review and publication

Preprints, conference papers, journal articles, registered reports, technical reports and theses have different review and dissemination arrangements. Peer review is scrutiny, not universal endorsement. Venue fit includes audience, contribution type, reporting requirements, rights and ethics—not just a prestige ranking. Make corrections visible after dissemination; publication does not freeze truth.

Use a small set of durable records rather than one endlessly growing conversation. A research brief holds the question and scope. A search log records retrieval decisions. A source note preserves exact identity, reading coverage and interpretation. A claim ledger connects conclusions to support and objections. A decision log explains changes. An artifact index points to actual outputs and versions. Keep private materials protected and separate from public evidence.

A workable rhythm

1. Orient: sharpen the question, contribution type and uncertainty.
2. Map: identify relevant communities, source channels and closest work.
3. Investigate: read, collect, build, interpret or prove according to the method.
4. Challenge: examine alternatives, errors and missing support.
5. Synthesize: update the argument and its boundaries.
6. Review: verify the output, obtain appropriate criticism and record the next obligation.

This is a revisitable loop, not a six-stage certification. Save a checkpoint at meaningful transitions: completed work, exact files/versions, unanswered questions, known failures, and the next feasible action. A checkpoint describing “done” without an output address is difficult to recover.

Stop, expand or revise?

Expand when a missing dimension could materially change the conclusion. Stop a bounded phase when its declared search/method has been completed and its remaining limits are explicit. Revise when the question, measure or assumptions no longer fit the evidence. A deadline can justify a scoped report, not an unrestricted completeness claim. Avoid “one more source” as a substitute for deciding what evidence would actually settle the next issue.

For a synthetic architecture example, move from “make research trustworthy” to a bounded question about stale-claim detection. Map prior work; define a stale acceptance event; compare against ordinary notes; include review overhead, false alarms and failure recovery; obtain consent for human testing; preserve planned versus observed outcomes. Until that study runs, the benefit is a hypothesis. This is a reusable reasoning pattern, not a reported experiment.