A Successful Response Is Not a Successful Result

One part of my day involves checking job websites out, auditing them, and generally trying to see whether they actually are useful or not. This is the second of three posts about that day and how I automated the process, and some of the issues I have run into running this n8n flow, self hosted on a small box. Rebuilding a Job Search That Had Stopped Working covered what was broken. This one is the reference piece: what the failure looks like, how to test for it, and what the twenty boards actually returned.

I spent a day probing more than twenty job sites to work out why two automated searches had stopped being useful. Eight of those sites accept a filter, answer normally, and ignore what you asked for. Nothing errors. Nothing warns. The response looks exactly like a filtered response, and it is not one. And this is increasingly the norm across job sites: a search that looks like it ran when it did not.

What is a decoy parameter?

A decoy parameter is a query string or filter that a site accepts, acknowledges and returns a successful response for, while having no effect on the result set. The status code is 200. The payload is well formed. The filter simply was never applied, and nothing in the response tells you so.

I found eight in a day, across sites that had no obvious relationship to each other.

How each one was confirmed

Every one was confirmed the same way, and it is the only way that works: change one parameter, run it again, and compare the two result sets. Not read the documentation. Not check the status code. Not trust the field name.

The rule

A successful response only proves the door opened. It proves nothing about whether your filter was applied. Change the parameter, then confirm the result set changed. If it did not, you are building on a number that is not real.

This had already cost me once, before I had this rule in place in my job search workflow. An earlier version of one search skipped its own location filter for a single source, on the reasonable assumption that the source’s own API had already done the filtering. It had not, and nothing about the responses said so.

The general form of the mistake is worth naming, because it is not really about job boards. A component that reports success is telling you about itself, not about your request. Any system that trusts an upstream filter without checking the output is carrying this bug quietly.

How to test for one

Four steps, and it takes about two minutes per parameter.

1. Run the query with the parameter set to a normal value. Save the result set.
2. Run it again with the parameter set to something wildly different, or removed entirely.
3. Compare the two result sets, not the status codes and not the response shape.
4. If they are identical, treat that as a failure signal, not a coincidence. Then check whether the first record is also identical, which usually confirms it.

The reason step four matters is that a genuine filter almost never returns a byte-identical set for two different inputs. When it does, you have found a decoy.

What twenty job boards look like in 2026

The pattern across the sweep was consistent and slightly damning.

The sites that are easiest to automate serve an industry most jobseekers are not in. I tested six remote work job APIs: Himalayas, RemoteOK, Remotive, Jobicy, Arbeitnow and Working Nomads. All reachable, all clean, all free, all documented. Across the 440 most recent jobs on all six combined, exactly one matched a learning design or training role, and that one was an English teacher on another continent. These are recruitment products for software engineers, and at that they are very good. For anyone else they are empty.

The sites carrying the roles I wanted are unglamorous sector boards. Teaching boards, university boards, charity boards, and one specialist recruitment agency, Instinct Resourcing.

The human filter

The single best source found all day was Instinct Resourcing’s vacancies page. One page, no search, no API, nothing to query. About eight of its twelve listings were on target, because a human specialist had already done the filtering that twenty automated boards could not. It was also the only source in the whole sweep to surface a live AI training vacancy.

Four other findings worth recording:

• One heavily promoted “no ghost jobs” board hides employer names and descriptions from anyone not signed in. The listings are real. You cannot see whose they are, or apply to them. That is a stated business model, rather than a defect, but it does mean the board cannot be verified independently, which is an odd position for a product whose pitch is verification.
• A major board’s RSS feed has ceased to exist. It had been the original search’s primary source.
• Several sites now serve a “Loading…” shell to anything that is not a full browser.
• Two recruitment consultancy domains resolve in DNS and never answer.

The incentive has shifted

What all this tells me, as the one looking for a job, is that right now there is money in email and data capture and in ring fencing information. Probably more than in the advertisements on the sites. This is what I have started calling monetisation decay: the site has shifted from helping you find a job to extracting value from your traffic. And honestly that is a whole other post right there.

What I would tell anyone building against this

Assume the filters are decorative until you have proved otherwise, and prove it per parameter rather than per site. Two of the eight decoys sat on sites whose other parameters worked correctly, so “this API is fine” is not a finding you can make once and reuse.

Then filter your own results anyway, locally, even when the upstream filter appears to work. It costs almost nothing and it is the only version you control.

This is post two of three. Post one is Rebuilding a Job Search That Had Stopped Working, on what was actually wrong with the two searches. Post three is The Parts the Machine Could Not Do, on checking advice, including the advice an AI gave me.