Hi! We see you’re using an ad-blocker. We’re fine with that and won’t stop you visiting the site.
But as we’re losing ad-revenue from this then why not make a donation towards website running costs?. Or you could disable your ad-blocker for this site. We think you’ll find our adverts are not overbearing!
Joined: Apr 04, 2006 Posts: 10118 Location: Bexhill, South Sussex, UK
Posted: Tue Jan 02, 2018 4:08 pm Post subject:
Privateer wrote:
However the Garmin search routine does not appear to be language-neutral so using the English keyboard you can't type "cafe" to find "Café", Which detracts from trying to force diacritical marks into the Garmin.
Does this fact not make the whole exercise pointless for Garmins then?
Joined: 30/12/2002 17:36:20 Posts: 4912 Location: Oxfordshire, England, UK
Posted: Wed Jan 03, 2018 3:23 pm Post subject:
M8TJT wrote:
Does this fact not make the whole exercise pointless for Garmins then?
Yes you're right, which is why I'd like Garmin to change it so that diacritical marks can be used in custom POIs and that you can search for strings regardless of letters having diacritical marks.
sussamb wrote:
I'm often in France and have had no issues searching for French addresses etc. Never tried cafe though so will do so later
Edit: Just checked and searching for cafe in France works fine ;)
Now that's very interesting. I've just tried on my Garmin 770 and searched for the attraction Hôtel del la Monnaie in Paris with the search string on "Hot" i.e. no diacritical marks and the hotel came up.
So, it would appear that built in POIs not only have diacritical marks but you can search for POIs without using diacritical marks.
I have retried my test favourite and have found that whilst you can add diacritical marks in a favourite, it seems that you can only search using the exact string i.e. use diacritical marks for the search if there are diacritical marks in the favourite name.
Now the question this poses is: Can diacritical marks be used in custom POIs like they are used in built-in POIs?
Looks like I need to re-read PhilHornby's post on using .GPX to see if I can get a test custom POI to a) show diacritical marks and b) search for text with/without using diacritical marks.
I've had a look at the Garmin built-in POIs for Jersey (Channel Islands) and they don't often use diacritical marks. Here are a few examples of my Jersey custom POIS that do use diacritical marks:
49.168060,-2.081460 Samarès Coast Hotel (Morvan) car park
49.181380,-2.241170 Corbière Lighthouse car park
49.173480,-2.078990 Samarès Manor
49.255740,-2.226860 Plémont Beach car park (top)
49.246220,-2.199640 Grève de Lecq Barracks
49.235990,-2.198900 Judith Quérée's Garden
49.180446,-2.230703 La Rosière Desalination Plant
49.251346,-2.226038 Café Ouen
49.246513,-2.199016 Le Câtel Fort
49.220129,-2.145919 Le Rât Cottage
Regards, _________________ Robert.
iPhone 6s Plus, iOS 14.0.1: iOS CamerAlert v2.0.7
TomTom GO Mobile iOS 2.3.1; TomTom (UK & ROI and Europe) iOS apps v1.29
Garmin Camper 770 LMT-D
Joined: Dec 07, 2006 Posts: 563 Location: North Devon
Posted: Wed Jan 03, 2018 6:10 pm Post subject: Experiments...
I have done some more experiments, using POILoader 2.7.3.0 on Windows 10. Test unit was a BMW-branded Nüvi 715.
This is what I believe to be to state of play, with regards to character sets available.
As previously stated, UTF-8 encoded CSV files are not accepted as input by POILoader.
GPX files that contain an "encoding=UTF-8" statement, must start with the ByteOrderMark /(BOM)/ 0xEF,0xBB,0xBF sequence to be accepted by POILoader.
GPX files read directly by the unit (i.e. as Favourites), must contain the "encoding=UTF-8" statement. The BOM is not necessary in this case!
Neither the unit, nor the .GPI file format support UTF-8!
Data in the .GPI file is stored as 8-bit ANSI (aka Windows-1252) data. If POILoader reads a UTF-8 encoded file containing a character that cannot be mapped into ANSI, it is replaced by a "?". If the unit itself reads such a GPX file, it silently ignores the entry.
So, if for example, a (properly) UTF-8 encoded .GPX file contains the sequence 0xE2,0x82,0xAC it is mapped to (and stored as) 0x80 in the .GPI file. (This is a Euro sign € and exists in both character sets.)
But the sequence 0xE2,0x84,0xA6 will be mapped to a "?" (0x3F0), as it is not present in the ANSI character set. (This is an Ohm symbol Ω)
As an aside - what character do you see on screen, in your POI generating software (Excel?), for the binary character 0x80? Is it the Euro sign € from the ANSI character set, or the Ä from the MacRoman set?
Joined: 30/12/2002 17:36:20 Posts: 4912 Location: Oxfordshire, England, UK
Posted: Wed Jan 03, 2018 8:34 pm Post subject: Re: Experiments...
PhilHornby wrote:
As an aside - what character do you see on screen, in your POI generating software (Excel?), for the binary character 0x80? Is it the Euro sign € from the ANSI character set, or the Ä from the MacRoman set?
Macs are frustratingly not like Windows, the Windows Character Map has no equivalent on a Mac, there are expensive apps that allegedly do a similar job but I've been had before so I don't really trust utility apps for the Mac.
Can I just confirm what you mean by "0x80", does the "x" represent a character A to F, or does it represent something else?
Sorry, it's been years since I've done any HEX and I just can't find the how to enter the "0x80" in the Mac or even how to Google to find the answer.
I'm using a Unicode Hex input virtual keyboard on the Mac and when I got to enter a character code in Mac Excel, I have the options of 0-9 and A-F, but no "x". Using A-F in 0x80 gives rubbish all the way through with neither € nor Ä.
Regards, _________________ Robert.
iPhone 6s Plus, iOS 14.0.1: iOS CamerAlert v2.0.7
TomTom GO Mobile iOS 2.3.1; TomTom (UK & ROI and Europe) iOS apps v1.29
Garmin Camper 770 LMT-D
Joined: Dec 07, 2006 Posts: 563 Location: North Devon
Posted: Thu Jan 04, 2018 2:39 am Post subject: Re: Experiments...
Privateer wrote:
Can I just confirm what you mean by "0x80", does the "x" represent a character A to F, or does it represent something else?
By 0x80, I mean 80hex i.e. the equivalent of 128 decimal. It's used because of the immense difficulty of adding a subscript of ₍₁₆₎ ... 80₍₁₆₎
What I'm trying to ascertain, is if you "see" the same interpretation of the data in your 'POI Editing' software, that the Nüvi does. (i.e. the 8 bit ANSI character set).
Then we need to see what is being stored in the file, when you hit save. For example, if you have a "€" Euro sign on screen and then store it in a file, does it write the UTF-8 3 byte sequence 0xE2,0x82,0xAC, or does it write a single 0x80 byte?
If it does the former - and you can't stop it - then you can't use a CSV file as input to POILoader - it doesn't understand UTF-8 CSV files. You would have to pass it via a GPX file, with the "encoding" denoted as "UTF-8" and (probably) with the BOM present.
(This would be a travesty, as the next thing POIloader would do is convert it back to ANSI (if possible), for inclusion in the .GPI file.)
Do you see where I'm coming from?
I've uploaded a CSV file to Dropbox, containing most of the 'special' characters in the 8 bit ANSI character set. (It was supposed to be all of them, but I went a bit cross-eyed, half way through!).
Joined: 30/12/2002 17:36:20 Posts: 4912 Location: Oxfordshire, England, UK
Posted: Fri Jan 05, 2018 7:36 pm Post subject:
Hi Phil,
I don't know how to insert a hex character 0x80 straight into Excel.
I use Excel worksheets (.xlsx) as my master file with a lot of columns:
ID; Latitude; Longitude; Name; Region; Categories; Phone; Created; Updated; Confirmed; Device; Name TomTom; Name Garmin.
Most columns are self-explanatory, the master list is a reference and basic change control:
ID is a simple Excel function "=ROW()-1" to give each POI a reference number for quick cross-checking.
Region, is not used at the moment but for my Jersey POIs it holds the name of the parish.
Categories: none; one; or more categories may be listed. They are a few characters such as Att (attraction); Food; JHT (Jersey Heritage Trust); etc to allow quick search for a type of POI in a single POI file.
Device: over the years I've used different SatNavs to capture POIs, more recently I use an app on my iPhone or even Google Earth.
Name TomTom: This contains a formula take the name, categories, ID, and telephone number and turn into a single string for use of TomTom.
Name Garmin: Similar to Name TomTom but doesn't use phone as Garmin handles that differently.
I manually take the Longitude, Latitude, Name (TomTom/Garmin), phone and create a .csv file which is Western (Mac OS Roman) encoded and has Windows (CRLF) line breaks. I can't remember whether BOM is used or not. The .csv is then passed through the Mac POI Loader.
I've downloaded your file and I will pass it through the the Mac POI Loader and then load it onto my Garmin Camper 770 LMT-D to see what the result is.
Regards, _________________ Robert.
iPhone 6s Plus, iOS 14.0.1: iOS CamerAlert v2.0.7
TomTom GO Mobile iOS 2.3.1; TomTom (UK & ROI and Europe) iOS apps v1.29
Garmin Camper 770 LMT-D
Joined: Dec 07, 2006 Posts: 563 Location: North Devon
Posted: Sat Jan 06, 2018 12:43 am Post subject:
Privateer wrote:
I don't know how to insert a hex character 0x80 straight into Excel.
You can use the "=CHAR" function...
i.e. =CHAR(HEX2DEC(80)) or just =CHAR(128)
If I do that (on Windows 10, using ancient Excel 2007), a euro sign (€) immediately appears in the cell. In other words, in the absence of any other instruction, Windows treats the 8 bit data as a "Windows-1252" character. My guess is, that on the Mac, you will get a Ä instead (this being 0x80 in the Mac Roman set). Various web sites are critical of Microsoft for sticking with the proprietary Apple stuff, when other software hasn't
If I save that file as a default "CSV", the file contains a "0x80" byte i.e. unchanged.
However, if I save that file as "CSV (Macintosh)", Excel writes the Mac Roman code for Euro sign to the file, which is"0xDB".
If I look at this "Macintosh" file in Notepad, or send it to a Nüvi, I now get a Û character instead of a € - because neither is expecting Mac Roman data - and has no way to detect it.
I found a cryptic reference to
Code:
the trial version of Apple Numbers from the Apple website
Apparently, this can
Code:
solve problematic issues encountered on Excel sometimes.
The suggestion being that you can import the CSV file and export in a different flavour.
Joined: 30/12/2002 17:36:20 Posts: 4912 Location: Oxfordshire, England, UK
Posted: Sun Jan 07, 2018 8:37 pm Post subject:
Oldboy wrote:
On a PC you would press, and hold, the Alt key while typing 080 on the numeric keypad.
It might be different on a Mac.
Thanks for that info. Yes, that sounds right as I’ve not used Windows for over a year now. You’re right, it is (very) different on a Mac.
PhilHornby wrote:
You can use the "=CHAR" function...
i.e. =CHAR(HEX2DEC(80)) or just =CHAR(128)
Thanks, that’s very useful to know. In Mac Excel 2016, I got the following results:
=CHAR(HEX2DEC(80)): Ä
=CHAR(128): Ä
This is all in a standard Mac Excel .xlsx workbook. Here are all of the .csv file types that you can save from Mac Excel. I have saved each .csv type and examined them with TextWrangler to show the details (i.e. encoding, line feed, and what the "Ä" is displayed as:
CSV UTF-8 (comma delimited) (.csv) --- Unicode (UTF-8, with BOM), Legacy MAC OS (CR) "Ä"
Comma Separated Values (.csv) --- Western (Mac OS Roman), Windows (CRLF) "Ä"
Windows Comma Separated (.csv) --- Western (Mac OS Roman), Windows (CRLF) "ƒ"
MS-DOS Comma Separated (.csv) --- Western (Mac OS Roman), Windows (CRLF) "é"
Joined: Dec 07, 2006 Posts: 563 Location: North Devon
Posted: Sun Jan 07, 2018 10:03 pm Post subject:
Oldboy wrote:
On a PC you would press, and hold, the Alt key while typing 080 on the numeric keypad.
Actually ... it's Alt0128, if we're being strictly accurate..
Privateer wrote:
In Mac Excel 2016, I got the following results:
=CHAR(HEX2DEC(80)): Ä
=CHAR(128): Ä
So here's the first issue. By default, MAC Excel is working in a different character set encoding to the Nüvi; it's using Mac Roman. You've got to stop it doing that (somehow), or use a different 'editor' that you can control.
Code:
CSV UTF-8 (comma delimited) (.csv) --- Unicode (UTF-8, with BOM), Legacy MAC OS (CR) "Ä"
Probably expected behaviour - but academic as POI Loader doesn't support CSV files encoded with a BOM.
Code:
Windows Comma Separated (.csv) --- Western (Mac OS Roman), Windows (CRLF) "ƒ]"
If it had left the value alone, it would be a start - it seems to have remapped it to something else. Exactly what and why, I can't figure out!
Privateer wrote:
I took the above file and processed it via Mac POI Loader and then loaded it onto my Garmin Camper 770 LMT-D. Here’s the result:
image link seems dead
This is more worrying...
it seems quite well documented that Mac Excel has its foibles, but Mac POI Loader seems to be getting in on the act too. It's almost as though it's expecting CSV files to be encoded using Mac Roman and has (sort of) translated their contents to Windows-1252. (Except it's mapped accented versions of the characters (as defined in Mac Roman), to their un-accented versions (as defined in Windows-1252).
aaaaaarrrrrrrrggggggghhhhhh
Since you may have to switch to GPX, to make this work, here are some more files for you to test with:
Joined: 30/12/2002 17:36:20 Posts: 4912 Location: Oxfordshire, England, UK
Posted: Mon Jan 08, 2018 11:42 am Post subject:
Hi Phil,
OK. I first loaded the V3.gpi file straight onto the Garmin, with the following results:
ANSI: Windows_1252 Encoding (V3.gpi file)
utf-eight: UTF-8 Encoding ¿ (V3.gpi file)
I then placed the ANSI.gpx and utf-eight.gpx into a folder for processing by the Mac POI Loader, with the following results:
ANSI: Windows_1252 Encoding (ANSI.gpx file)
utf-eight: UTF-8 Encoding (utf-eight.gpx file)
My observations
I'm presuming that your v3.gpi file was processed by Windows POI Loader.
.gpx files (ANSI or UTF-8) that are processed by Windows POI Loader, appear to show characters correctly.
The Mac POI Loader doesn't seem to be able to process diacritical marks or extended character sets either by ANSI or UTF-8 .gpx files.
It really does look like the Mac version of POI Loader is flawed.
One other thing Phil, I noticed that the name of the POI itself was either UTF-8 Encoding or Windows_1252 Encoding, with the diacritical marks wishing the POI description or address. Would it be possible to try another test with several POIs within one file, each with a diacritical mark in the actual name? As this would allow us to test whether the Garmin can search for a POI name that has an accent by using the English keyboard that doesn't have didactical marks, i.e. search for "Café" by using the search string "cafe".
Many thanks, _________________ Robert.
iPhone 6s Plus, iOS 14.0.1: iOS CamerAlert v2.0.7
TomTom GO Mobile iOS 2.3.1; TomTom (UK & ROI and Europe) iOS apps v1.29
Garmin Camper 770 LMT-D
Joined: Dec 27, 2006 Posts: 998 Location: South Lincs, UK.
Posted: Mon Jan 08, 2018 1:52 pm Post subject:
Privateer wrote:
...this would allow us to test whether the Garmin can search for a POI name that has an accent by using the English keyboard that doesn't have didactical marks, i.e. search for "Café" by using the search string "cafe".
I have tried this on Windows by converting the list of POIs that you posted earlier in this thread to a GPI file using the Windows version of POILoader. The Garmin doesn't find Café Ouen when searching for Cafe. I also tried translating the Diacritical marks into their basic equivalents and putting this into the fourth column of the CSV file e.g.
Code:
49.251346,-2.226038,"Café Ouen","Cafe Ouen"
This results in "Cafe Ouen" appearing in the POI description of the POI named "Café Ouen" but the Garmin still doesn't find it when searching for Cafe. _________________ Paul
Joined: Dec 07, 2006 Posts: 563 Location: North Devon
Posted: Mon Jan 08, 2018 3:59 pm Post subject:
Privateer wrote:
My observations
I'm presuming that your v3.gpi file was processed by Windows POI Loader.Yes
.gpx files (ANSI or UTF-8) that are processed by Windows POI Loader, appear to show characters correctly.
Yes
The Mac POI Loader doesn't seem to be able to process diacritical marks or extended character sets either by ANSI or UTF-8 .gpx files.
It really does look like the Mac version of POI Loader is flawed.
Sadly, yes that appears to be the case. While you could make the argument that with CSV input, it should default to expecting Mac Roman encoding (on a Mac), no such get-out exists for GPX input. The GPX files are properly formed and encoded. They are quite specific as to the encoding of the data. They use the same character set that the Nüvi uses, so there is really no excuse for mangling them into something else.
Would it be possible to try another test with several POIs within one file, each with a diacritical mark in the actual name?
I'll knock something together shortly
pcaouolte wrote:
Privateer wrote:
...this would allow us to test whether the Garmin can search for a POI name that has an accent by using the English keyboard that doesn't have didactical marks, i.e. search for "Café" by using the search string "cafe".
I have tried this on Windows by converting the list of POIs that you posted earlier in this thread to a GPI file using the Windows version of POILoader. The Garmin doesn't find Café Ouen when searching for Cafe. I also tried translating the Diacritical marks into their basic equivalents and putting this into the fourth column of the CSV file e.g.
Code:
49.251346,-2.226038,"Café Ouen","Cafe Ouen"
This results in "Cafe Ouen" appearing in the POI description of the POI named "Café Ouen" but the Garmin still doesn't find it when searching for Cafe.
I think that is in keeping with what I've seen so far, as well.
Posted: Today Post subject: Pocket GPS Advertising
We see you’re using an ad-blocker. We’re fine with that and won’t stop you visiting the site.
Have you considered making a donation towards website running costs?. Or you could disable your ad-blocker for this site. We think you’ll find our adverts are not overbearing!
All times are GMT + 1 Hour Goto page Previous1, 2, 3Next
Page 2 of 3
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
Or you could disable your ad-blocker for this site. We think you’ll find our adverts are not overbearing!
Hi! We see you’re using an ad-blocker. We’re fine with that and won’t stop you visiting the site.
But as we’re losing ad-revenue from this then why not make a donation towards website running costs?. Or you could disable your ad-blocker for this site. We think you’ll find our adverts are not overbearing!