Feedsaz Internet Solutions terméke

How RSS Feed Creator Removes Duplicates in Merged Feeds

2026. november 16.8 perc olvasásRSS-feedek
How RSS Feed Creator Removes Duplicates in Merged Feeds

Short answer: When RSS Feed Creator builds a feed, it keeps one item for each web address. Two items count as the same when their links are the same after capital letters and a trailing slash are ignored. The first copy it meets, in the order of your sources, is the one kept, and the rest are dropped before the newest-first sorting and the item limit are applied. Items that point to different addresses, such as a story syndicated on two websites, are different items and both appear.

Why duplicates appear in the first place

As soon as a feed is built from more than one source, overlap is almost certain. A company announces something on its blog and again on its press page. A news site lists the same story in a section page and on its home page. A category page shows posts that the main blog list also shows. Even a single source can repeat an item, for example when a “featured” block sits above the list and again inside it.

Duplicates are annoying in a reader and costly in an automation. A posting tool that sends two copies of the same story to social profiles looks careless. For that reason, RSS Feed Creator removes duplicates during every build, without any setting to switch on. The merge feature itself is introduced in merge sources in RSS Feed Creator: up to 10 in one feed.

It also matters for people who read feeds in more than one place. If the same feed address is used in a reader and in a posting tool, both will see the same cleaned list, which means the duplicates you would otherwise have to remove by hand in each tool never arrive in the first place. That is a quiet but real saving of attention.

What counts as the same item

The rule is simple and predictable: an item is identified by its link. Before comparing, the tool turns the link into lower case and removes a closing slash. So these count as the same item:

This is the same idea as the item identifier in a feed. RSS Feed Creator writes the link as the identifier of every item it produces, so readers also treat an unchanged item as one that they have already seen. For a deeper look at how identifiers behave in readers, RSS GUIDs explained: why readers show old posts as new is a good reference.

Which copy is kept

Sources are read in the order in which they appear on the management page. The first time an address is met, the item is kept, together with its title, summary, picture and date from that source. Later copies with the same address are dropped. That has a practical consequence: if two sources describe the same item differently, the first source wins.

So put your preferred source first. If one source gives better summaries or pictures, for instance an official press page versus a news site, make sure it is higher in the list. Sources appear in the order in which you added them, and removing and re-adding a source is how you move it down.

One more detail: the filters are applied before an item is marked as seen. If the first copy contains a word from your skip box but the second copy, from another source, does not, the second copy is the one that appears. That is rarely a concern, but it explains odd cases.

What is not treated as a duplicate

Because the comparison is on the address, anything that changes the address makes a new item. The cases you will meet most often are:

Situation Why it appears twice
http and https versions The addresses differ at the start
With and without www The host name differs
Tracking parameters such as utm_source The query string differs
The same story on two websites Different publishers, different addresses
Print or mobile versions of a page Different paths

None of these is a fault. The tool cannot know that two different addresses lead to the same story, and guessing would risk hiding items you want. It is better to show both and let you decide.

How to avoid the common duplicates

Most avoidable duplicates come from sources chosen carelessly. Four habits remove nearly all of them:

  1. Choose list pages that do not overlap. Do not merge a blog with one of its own category pages, because the category items also appear on the blog.
  2. Use clean addresses. Paste the plain address of the page, without tracking parameters, so the links read from it are the site’s own.
  3. Prefer an official feed when there is one. Official feeds usually carry the canonical address of each item. The preview tells you when a page announces a feed.
  4. Use the skip box for syndicated copies. If a partner site always adds a phrase such as “originally published on” to its copies, put that phrase in the skip box.

Checking the sources in the preview before adding them, as described in the RSS Feed Creator preview, shows the addresses of the items and makes overlaps visible early.

The order of operations in a build

It helps to know what happens when a feed is built, step by step. First the tool reads each source in order. Then it goes through all items: it skips an item whose address was already seen, applies the include and skip words and only then records the address. If the feed has more than one source, the remaining items are sorted with the newest first. Finally the list is cut to the item limit of your plan: up to 20 on the free plan and up to 100 on paid plans.

The order has two useful effects. Duplicates never take up space in the limit, so a merged feed holds the most distinct items it can. And a strict filter reduces the number of items before the limit is applied, so you see more of what you care about rather than more noise.

Check a merged feed in two minutes

After you add sources, open the management page and read the latest items. Look for two kinds of repetition: identical titles that should have been merged, and different titles about the same event. For the first, compare the addresses; if they differ only by a tracking parameter or by http and https, change the source address. For the second, nothing is wrong: you are seeing two publishers’ views of one event, which is often valuable in monitoring.

If duplicates remain a nuisance in your reader or a posting tool, consider removing the overlapping source. Monitoring hundreds of sources with RSS without chaos has ideas for keeping large source lists tidy.

Three real-life examples

A company blog and its press page. A manufacturer posts each announcement on its blog and again, with the same link, in a press section. You merge both. The announcements appear once, with the details of whichever source you added first. If you prefer the press page’s wording, put it first.

A news site with a home page and a section page. The home page shows a handful of stories from every section, and the section page shows all stories of one section. If you add both, the overlap is removed automatically, but the home page adds items from sections you do not care about. Keep only the section page, and your feed is both smaller and cleaner.

A story reported by two outlets. A regulator announces a decision, and two trade sites cover it. You see three items, one from each source, because their addresses differ. This is useful in monitoring, since the trade sites add context. If you only want the original, add the regulator’s page and not the trade sites, or put words that appear only in the trade coverage into the skip box.

In each case the answer is a small change to the list of sources rather than a new setting. Because the feed is rebuilt when you save, you can try the change and look at the result at once.

Plans and limits

Duplicate removal works on every plan. Merging several sources into one feed belongs to the paid plans: the PostRSS add-on at €5 per month allows up to 10 sources in a feed, with 10 feeds, up to 100 items each and a 30-minute refresh. The free plan has one source per feed and up to 20 items, and a 3-hour refresh. The details are on the pricing page.

Related reading

The bottom line

RSS Feed Creator keeps one item per web address in every build, favours the first source in your list and applies your filters before it counts items against the limit. Choose non-overlapping sources with clean addresses and expect the occasional story to appear from two publishers.

GYIK

Do I need to switch duplicate removal on?

No. Duplicates are removed automatically every time a feed is built.

Which source wins when two sources have the same item?

The first source in your list. Its title, summary, picture and date are kept.

Why do I see the same story twice?

The two items probably have different addresses, for example different websites, http and https, or a tracking parameter.

Are capital letters or a trailing slash a problem?

No. They are ignored when addresses are compared.

Does merging work on the free plan?

Merging several sources is part of the paid plans. The free plan has one source per feed.

#Content curation#Feed Troubleshooting#Keyword filters#RSS Feed Generator
Hozza létre első feedjét — ingyen.Feed bármely oldalból. Minden termék minden katalógusban.
Kezdje ingyen

Továbbiak a blogról

Összes cikk →
Internet Solutions

Továbbiak csapatunktól

Az Internet Solutions fejlesztése. Próbálja ki többi termékünket is — mindegyik másképp spórol Önnek időt.

internet-solutions.net ↗