Accessing iRecord/Indicia records in the DDb

[admin]

[FIRST DRAFT]

Most iRecord data (up to ~ May) is now accessible in the DDb. To access these records set a 'project' filter when searching ('more options' => 'source' => 'project workspace' --> 'indicia').

The records are partitioned away separately from the main DDb workspace but can be transferred across. To do that select records and using the options at the bottom of the search page choose 'transfer in to DDb workspace'. Alternatively, for these records there is a tick box available when 'confirming' them to say that you also want the record added to the DDb.

Until transferred to the main DDb workspace iRecord records would not appear on maps and wouldn't be used for the Atlas.

Records that have been transferred to the DDb will still also continue appear in the 'Indicia' workspace. You can filter-out data that has already been transferred using 'more options' => 'admin' => 'compare workspace' (choosing 'other branch present' or 'absent')

The full, up-to-date iRecord data set (also including iNaturalist records) will be added soon. The process for transferring back validation changes to iRecord will also be in place soon (meanwhile feel free to validate iRecord records in the DDb, those validations will also reach iRecord once the full process is up and running).

Note that the criteria for iRecord data can differ from usual BSBI standards (e.g. records from gardens might be included). In general please feel free to 'confirm' such records if they are valid, but avoid transferring them to the DDb. Records from 'Garden Bioblitz's have been be pre-flagged as do-not-map for DDb purposes.

It will not be possible to edit iRecord data in the DDb (please avoid attempting to, as the process to prevent edits may not be completely robust at the moment). The expectation is the iRecord data that needs correcting should be changed at source in iRecord.

You can view the iRecord version of a record by following the linked DDb record's external id.

I would welcome feedback.

[AndyAmphlett]

Tom,

Thanks for that update. I have made a start on vc94 / vc96 Indicia records.

One issue I have noticed is that there are records of Geranium himalayense, which when looking at the record details appear to originally refer to Geranium pratense - https://database.bsbi.org/search.php#re ... dce3a16d54. The latter is likely, the former not. It looks like the taxon dictionary used in (for example iSpot) may have an error?

I will feedback some general comments about validating Indicia records once finished.

Andy.

[AndyAmphlett]

Tom,

There are a set of records for vcs 94 and 96, that had been accepted by automatic checks within Indicia. I was happy that those records were ok, so moved them to the main DDb workspace. But in the DDb workspace they appear as unchecked. Should anything being moved to the main DDb workspace always be confirmed at the same time, even if already checked within Indicia? The records in question have now been confirmed within the DDb.

Thanks,

Andy.

[AndyAmphlett]

Verification & validation of Indicia records on the DDB

I hope the following, based on my experience of validating Indicia records for two Scottish vcs will be of interest / help.

There are records from Indicia for all vcs apart from 6 vcs in Ireland (H) 11, 13, 22, 24, 26 and 32. Number of records per vc varies from 1 to 37,983 (median = 2751).

I have checked the records for vc94 and vc96 (2906 records in total). I undertook all the verification and validation within the Indicia workspace in the DDb, before the records were moved into the main DDb workspace. I split the records by dataset, and further subdivided them by taxon / recorder / grid reference etc as appropriate. Some vcs have a lot of records to check, eg. 48 vcs have >5000 Indicia records. To make the process of validation efficient make sure you use the DDb's validation tools, eg. to find hectad singletons. Familiarity with using the DDb, and some experience of validation are both required before tackling the Indicia records.

I ignored any records from Garden Bioblitz datasets (as being potentially outwith BSBI recording guidelines), as well as any records where there was no proper recorder name (mainly from the iSpot dataset). I also ignored records where the site name was invalid (typically an address that did not match the site grid reference), but did accept records with no location name. A number of records in the BSS Urban Flora project dataset had obviously corrupted sequences of grid references (perhaps errors created in a spreadsheet prior to import).

Some records could have been accepted if the taxon had been recorded as an aggregate, rather than one of the segregates that were unlikely to be correct for the vcs in question. For example records of Gymnadenia conopsea s.s. would be acceptable as the sens. lat. taxon, but not as the s.s. taxon (which does not occur in these vcs). Guidance is not to edit Indicia records within the DDb, so these were marked as 'needs checking' with a reason given.

Many potential new hectad records were also marked as 'needs checking', especially when they were a marked extension of range and the recorder was unknown to me. Few records have supporting details, but some do have photos which are often very useful, supporting or contradicting a record's ID. NB some of the record identifications are, from looking at accompanying photos, spectacular errors.

In total I accepted (confirmed and moved to the main DDb workspace) 91.5% of the vc94 and vc96 records. In this sample of records, the 'error' rate (records not acceptable for Atlas 2020) was approaching 10%, which is, in my experience, very high. Typically, eg records via record cards from experienced recorders, the error rate will be 0.5% or less. Therefore, I suggest that the Indicia records for other vcs will require very careful checking before inclusion in the main DDb workspace. It must be stressed that the Indicia records appear to have a much higher error rate, and proportion of records unsuitable for inclusion in the main DDb workspace, than VCRs may be used to dealing with.

On the positive side, the vc94 and vc96 records did contain a small number of potentially really interesting new records, but they will mainly require additional supporting information, or field visits to confirm. Hence they will be unlikely to make it into the final Atlas 2020 dataset. This is the primary frustration with the Indicia records; potentially good new records by recorders unknown to the VCR, and with no means of directly contacting them to raise any queries, and no supporting evidence.

I noticed that some records had been accepted by automatic checks within Indicia. I was happy that most of those records were OK, so moved them to the main DDb workspace. But in the DDb workspace they appear as unchecked, so still require confirmation. NB out of about 400 records accepted by the automatic checks within Indicia, I found 3 or 4 that I was not happy to accept.

Andy.

[petestroh01]

That is all very useful, thank you Andy. I will pass on your comments to England VCRs, and also refer to this message board 'conversation'.

[jdsh]

Well, I've had a quick glance at the vc29 records.
There are certainly some exceedingly unlikely species being recorded. Unfortunately these are mixed in with perfectly plausible records by the same recorder, so these then have to be regarded as suspect as well.
Some records are duplicates, where the recorder has followed the normal procedure of submitting them to the county recorder and so should already be in the DDb.
Some are not duplicates, but the recorder hasn't sent them to the county recorder, even when the recorder should know that this is the expected procedure.
There are a lot of common species.
Some grid references are of unjustifiable precession - 10 figure references are only reliable if done with differential GPS and certainly not necessary for Crataegus monogyna for example.
Overall I don't think I would consider importing these records to be a high priority, but if I was going to do so would want them in my local MapMate database first. I'd probably clean them locally first by checking for RPPC or county scarce alien species and then ruling out some recorders before importing the records.

Jon

[AndyAmphlett]

Tom,

I notice that some iSpot records do not automatically display a photo within the DDb, while some do. I was puzzled when some iSpot records, confirmed by online determiners lacked a photo. I checked, and for example this DDb record (a new hectad record for Linnaea borealis) - https://database.bsbi.org/record/2cd4p9h.ge67np has no photo on the DDb**. But tracking the record down in iSpot it has two photos and is correctly identified. See https://www.ispotnature.org/communities ... land-plant. If I go to the iSpot external ID link given in the DDb version I don't see any photos***, which gives the impression that there are none, which is incorrect.

Not sure what is going on here, but assuming all iSpot records do have photos, then being able to view them within the DDb would allow verification to take place.

Andy.

Edit. ** that link does not work. Record can be viewed at https://database.bsbi.org/search.php#re ... f95c7bfefd
Edit. *** the external ID link for the record ID does not show a photo, but searching on the iSpot link included in the record details does.