Internal Format of XDL files

Discuss anything TunerPro related.

Moderators: Mangus, robertisaar, dex

Post Reply
razorx
Posts: 5
Joined: Mon Mar 24, 2014 10:16 am

Internal Format of XDL files

Post by razorx »

Hi,

I am working on a micro-controller board to write logs to a SD/MMC card.
More Easy than taking your laptop with you. ;)

It has already built into my car and captures the ECM/ECU data. I only need to have the internal format of the XDL files, so I could make the micro-controller write down the file in XDL format.

Any information on this would be highly appreciated.
ReadyXB
Posts: 8
Joined: Thu May 29, 2014 12:42 pm

Post by ReadyXB »

Did you figure out the format? I have decoded most.
razorx
Posts: 5
Joined: Mon Mar 24, 2014 10:16 am

Post by razorx »

ReadyXB wrote:Did you figure out the format? I have decoded most.
Wow!

Would you like to share this? This is very handy for anyone setting up a microcontroller based logging system.


.I haven't come any further with this, I am very busy with a semi automatic VE map calculator sheet based on TunerPro's history tables. I am convinced that I can make the same handy sheet for the TCV duty cyle pilot control maps. Saves a lot of time.

Driving a Volvo V70R. Motronic 4.4. ;)
ReadyXB
Posts: 8
Joined: Thu May 29, 2014 12:42 pm

Post by ReadyXB »

razorx wrote: Wow!

Would you like to share this? This is very handy for anyone setting up a microcontroller based logging system.
I agree. I have a PM out to Mangus, but he has not yet replied, even though he seems to be replying to forum posts. I want to a) confirm a few unknowns, and b) be sure he is ok with the details being posted. I imagine the only concern would if the .XDL format changes in future Tunerpro revisions, it might breaking outside software.
User avatar
Mangus
TunerPro Author
Posts: 1918
Joined: Wed Mar 19, 2003 1:49 pm

Post by Mangus »

ReadyXB wrote:I imagine the only concern would if the .XDL format changes in future Tunerpro revisions, it might breaking outside software.
Exactly correct. I can share the details with you, but I'm about 90% of the way done with replacing the entire logging system with a new, custom database-like implementation. Due to my own time constraints (I'm sure everyone has noticed that releases are further between and less feature-rich), I can't tell you exactly when the replacement will roll out, but it will happen.
***************************************
TunerPro Author
1989 Trans Am
razorx
Posts: 5
Joined: Mon Mar 24, 2014 10:16 am

Post by razorx »

Mangus wrote:
ReadyXB wrote:I imagine the only concern would if the .XDL format changes in future Tunerpro revisions, it might breaking outside software.
Exactly correct. I can share the details with you, but I'm about 90% of the way done with replacing the entire logging system with a new, custom database-like implementation. Due to my own time constraints (I'm sure everyone has noticed that releases are further between and less feature-rich), I can't tell you exactly when the replacement will roll out, but it will happen.
OK thanks,

In the meantime I use an Asus Transformer book. To do the logging.
Controlled via an RDP session by an double din Parrot Asteroid Smart. Works fine, buy I want to make daily drive logging more simple. What car and engine manangement do you have?
festy
Posts: 25
Joined: Thu May 17, 2012 3:25 pm
Location: Oz

Post by festy »

Mangus wrote:
ReadyXB wrote:I imagine the only concern would if the .XDL format changes in future Tunerpro revisions, it might breaking outside software.
Exactly correct. I can share the details with you, but I'm about 90% of the way done with replacing the entire logging system with a new, custom database-like implementation. Due to my own time constraints (I'm sure everyone has noticed that releases are further between and less feature-rich), I can't tell you exactly when the replacement will roll out, but it will happen.
Are you planning on publishing the new format, or at least still support reading XDL format files as well?
My raspberry pi based ALDL dash uses your ADX/XDL formats for logging, it's very handy to be able to pop a USB stick out of the dash and open straight up in Tuner Pro...
razorx
Posts: 5
Joined: Mon Mar 24, 2014 10:16 am

Post by razorx »

Mangus wrote:
ReadyXB wrote:I imagine the only concern would if the .XDL format changes in future Tunerpro revisions, it might breaking outside software.
Exactly correct. I can share the details with you, but I'm about 90% of the way done with replacing the entire logging system with a new, custom database-like implementation. Due to my own time constraints (I'm sure everyone has noticed that releases are further between and less feature-rich), I can't tell you exactly when the replacement will roll out, but it will happen.
Take your time ;)

Offtopic: The history tables are perfect for semi automatic map corrections in Excel. Works very nice.

A database based logging would be perfect. Giving the user the opportunity to collect useful data and do the calculations on it even more easily.
User avatar
dfddfd2
Posts: 112
Joined: Wed Feb 27, 2008 4:37 pm
Location: Burgaw, NC

Post by dfddfd2 »

Rather than have the micro-controller board store data in XDL format, You might try storing the raw data on the SD card, then emulate the ECU side of data logging back to TunerPro. This way, it doesn't matter what the XDL format is. Your micro-controller board is already emulating the TunerPro side of the logging protocol to read the data. The only issue you may have is the per-packet relative time stamp, which means you may have to pace the communications back to TunerPro so it time stamps as if it was actually talking to the ECU.
razorx
Posts: 5
Joined: Mon Mar 24, 2014 10:16 am

Post by razorx »

dfddfd2 wrote:Rather than have the micro-controller board store data in XDL format, You might try storing the raw data on the SD card, then emulate the ECU side of data logging back to TunerPro. This way, it doesn't matter what the XDL format is. Your micro-controller board is already emulating the TunerPro side of the logging protocol to read the data. The only issue you may have is the per-packet relative time stamp, which means you may have to pace the communications back to TunerPro so it time stamps as if it was actually talking to the ECU.
Good suggestion!
festy
Posts: 25
Joined: Thu May 17, 2012 3:25 pm
Location: Oz

Post by festy »

The problem with doing it that way is that an hour long log would then take an hour to transfer to TP :cry:
The XDL format is pretty much just a timestamp plus the data returned by the ECU anyway - if you compare a hex dump of an xdl file with it's csv export, you should be able to spot the record structure.
If you use a short log with the engine off (so pretty static data values) it's even easier.
ReadyXB
Posts: 8
Joined: Thu May 29, 2014 12:42 pm

Post by ReadyXB »

dfddfd2 wrote:Rather than have the micro-controller board store data in XDL format, You might try storing the raw data on the SD card, then emulate the ECU side of data logging back to TunerPro. This way, it doesn't matter what the XDL format is. Your micro-controller board is already emulating the TunerPro side of the logging protocol to read the data. The only issue you may have is the per-packet relative time stamp, which means you may have to pace the communications back to TunerPro so it time stamps as if it was actually talking to the ECU.
I had implemented this method already and it does work. The time factor is currently unavoidable (although there are options that diags software could implement to eliminate this), but still, it gives me flexibility to continue logging if new diags software and/or log formats are released and I'm trying to play catchup.
Mafketel
Posts: 4
Joined: Sat Apr 18, 2015 2:37 pm

Its 2016 now xdl is still used ;)

Post by Mafketel »

I know this is an old tread but the question is exactly right but sadly no answer.

So can I conclude that the format of the logfile is not going to be available?

It is about 2 years later and the logfile seems not to have changed.
From what i have gathered
the first 1 line info from tunerpro in my case 146 bytes long lotsa FF's on the end, does this change between different sessions?
next lines
first 12 bytes tunerpro info consisting of:
a timestamp(first 8 bytes) I am not entirely sure what is what within.
first 4 bytes are not clear yet
last 4 bytes LSB in ms
9 byte always 1? as a delimiter?
10 11 12 byte I don not know

the raw info from the ECU(in my case 135 bytes)

next entry in the logfile starting with the timestamp.

At least this is what i can gather from the file i have from the ecu log with 135 bytes of data from the ecu. And repetition at 147 bytes line length.
Last edited by Mafketel on Wed Sep 13, 2017 5:08 am, edited 1 time in total.
Mafketel
Posts: 4
Joined: Sat Apr 18, 2015 2:37 pm

It was not an aprils fool question

Post by Mafketel »

A small bump ;)
festy
Posts: 25
Joined: Thu May 17, 2012 3:25 pm
Location: Oz

Post by festy »

I don't believe there are any plans to publish the XDL format, but regardless of whether or not it remains the primary log format used by Tuner Pro I think it's safe to assume it will continue to be supported in it's current form for the foreseeable future.
Mafketel
Posts: 4
Joined: Sat Apr 18, 2015 2:37 pm

Post by Mafketel »

Is there any good reason not to publish the format?

eventually somebody will share most details and it is open. Untill then ppl wanting to know just have to separately invest the time to decode.

I really would like not to spend more time figuring something out i do not have to.

I prefer just write some code on the microcontroller that also takes into account the crc in the datastream and not have to take my laptop to log data.

just read a different ecm and the header is still 146 bytes long even though the data line repetition is 115 bytes
Mafketel
Posts: 4
Joined: Sat Apr 18, 2015 2:37 pm

Post by Mafketel »

a small bump ....

maybe someone would like to shine a light on the format?
Acki
Posts: 28
Joined: Wed Mar 06, 2013 12:00 pm

Re: Internal Format of XDL files

Post by Acki »

Any update her? A CSV -> XDL converter tool would be awesome!
Post Reply