Hypothesis: learners who get a hint after their second wrong answer finish the module more often than learners who get no hint. That sentence is falsifiable; it names one variable and one outcome you can count. Writing claims like it, then building the plumbing to check them, is most of this role. Most remote learning engineer jobs describe the work in general terms; this posting walks through one test from start to finish so you can judge whether it is the kind of week you want.
Full-time, fully remote, and open in every country, the position pays 80,000 USD per year. What follows is the cycle you would run, in the order you would run it.
Advertisement
Start with a prediction grounded in learning theory
The hint idea does not come from a hunch. Research on productive struggle suggests that a small amount of difficulty helps people remember, while repeated failure with no support pushes them to quit. Two misses felt like a reasonable line between the two. You would write that reasoning down, state the expected direction of the effect, and pick the metric before touching any data: module completion within seven days of the first attempt, for example.
Writing the metric first matters. If you pick it after seeing results, you will find something that looks significant, and it will not replicate. Part of this learning engineer job description is being the person on the team who insists on that order.
Instrument the course with xAPI statements
A course that only reports "completed" or "not completed" cannot answer the question. So you add xAPI statements at the moments that matter. Each attempt on a quiz item sends an actor, a verb such as
answered, the item as the object, and a result block holding success and the response, with the attempt number carried in an extension. When the hint appears, a separate statement with the verb
experienced and an extension like
hint_variant: "B" records which arm the learner was in.
Then you test it. You open the learning record store, take the quiz yourself three times with wrong answers, and confirm that the attempt counter increments and the hint statement fires exactly once. Many experiments fail here because a statement fired twice on refresh or never on mobile. Catching that before launch is cheaper than explaining a broken dataset afterward.
Pull the LMS data with SQL
Once the test has run long enough, you query the LMS tables and the statement store. A typical pull joins enrollments to xAPI attempts, filters to learners who reached the second wrong answer on at least one item (the only group the hint could affect), and groups by variant. Expect window functions for first attempt dates, and unglamorous joins where a user ID is text in one table and an integer in another.
In roles like this, access is often granted per project, usually as read access to reporting replicas rather than production databases, with learner data pseudonymized. How this employer handles it isn't stated; it is a fair question for the first call.
Analyze it in Python
The SQL output lands in a pandas DataFrame. From there, check the sample ratio first: if the split was meant to be 50/50 and you see 58/42, something in the assignment broke, and you can't trust the result until you find out why. After that, include completion rates per arm, a confidence interval on the difference, and a look at drop-off points by item so you can see where people actually stop.
You would also break results down by device and by prior course history, while being honest that those cuts were exploratory. A notebook that a colleague can rerun from top to bottom is the expected deliverable, not a screenshot of a chart.
Read the A/B result and decide what changes
Suppose the hint arm finished more often, but the gain came almost entirely from two hard items in the middle of the module. That is a more useful finding than a single headline number. It suggests rewriting those two items, keeping the hint, and running a follow-up test on wording. Suppose instead there was no difference. Then you report that plainly, note the sample size and the smallest effect the test could have detected, and propose what to try next.
Either way, the results go into a short written summary for designers and engineers, followed by a decision. Shipping improvements, then measuring again, is the loop.
Learning engineer and instructional designer, side by side
An instructional designer usually owns the learning experience itself: objectives, sequencing, the scripts and activities, the assessments. A learning engineer works on the same course but treats it as a system that produces evidence. You design activities from learning science too, yet you also decide what gets logged, write the queries, run the statistics, and argue for changes using data.
In practice, the two roles sit side by side. The designer might propose three ways to explain a concept; you would set up the comparison and tell them which one helped. If you are an instructional designer thinking about switching, the gap is mostly technical. Learn enough SQL to write joins and aggregates comfortably, learn Python for data cleaning and basic statistics, and instrument one of your own courses with xAPI. That project is most of the answer to how to become a learning engineer.
Course builders with SQL: the profile for remote learning engineer jobs
Required: a degree in learning sciences, instructional design, cognitive science, or data science, plus two to four years building digital courses or doing analytics work. Working knowledge of Python and SQL is required, as is hands-on time with xAPI and LMS data. You should be comfortable explaining why a spacing effect or worked examples might help, and equally comfortable explaining what a p-value does and does not tell you.
With two to four years asked for, this post won't suit searches for entry-level remote learning engineer jobs, and anyone hunting for learning engineer jobs no experience required will find it a stretch. Someone with two years of course building and one serious analytics project, though, would be a reasonable applicant.
The portfolio that gets read
A portfolio of instrumented learning products is expected. That does not mean polished screenshots. The strongest submissions show one product end to end: the question you asked, a sample of the statements you logged, the query, the notebook, and what you changed afterward. A null result written up honestly counts. So does a small personal project, such as a flashcard app where you logged response times and tested two review schedules.
On your learning engineer resume, put that project near the top and describe outcomes in terms of decisions made, not tools listed.
80,000 USD per year, from any country
Here, the learning engineer salary is set at 80,000 USD per year, full-time. No market ranges are given here. Since applicants from every country are eligible, it shows up among learning engineer jobs remote worldwide and learning engineer work-from-home jobs. Collaboration tools go unnamed. Remote teams doing this work commonly rely on shared documents, version-controlled notebooks, a ticket board, and video calls; find out how working hours overlap across time zones.
Questions that test the full experiment cycle
Expect learning engineer interview questions that test the full cycle. You might be given a small table of attempts and asked to compute completion by variant, or asked how you would know a test was underpowered. You may be asked to design xAPI statements for a simulation, or to explain why a correlation between video watching and grades does not prove the videos work.
To learn learning engineering skills more broadly, free statistics material, the published xAPI specification, and building projects will take you further than any single learning engineer course or certification. Credentials are welcome, but none is required here.
One experiment, written up, as your cover note
Among remote learning engineer jobs hiring now, this one costs nothing to apply for. Along with your resume and portfolio link, write a few sentences on one experiment you ran, including what went wrong. If anyone in a hiring process asks you to pay, treat it like a sample ratio mismatch: something is broken, and the result cannot be trusted.
