Nightcrawler's Message Board
http://transcorp.romhacking.net/forum/YaBB.pl
Rom Hacking/Translation Board >> Rom Hacking/Translation Board >> Project - Text Angel (GUI Generic Dumper/Inserter)
http://transcorp.romhacking.net/forum/YaBB.pl?num=1273690996

Message started by Nightcrawler on May 12th, 2010 at 3:03pm

Title: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on May 12th, 2010 at 3:03pm
Overview:
In pursuit of working smarter and not harder I had started toying around with the idea of new GUI based generic dumper/inserter. The aim for the dumper was originally be able to do everything that Cartographer could do and then some. I was trying to eliminate the need for a custom dumper in as many cases as possible. On the insertion side, it would ideally be able to insert everything that it can dump. I thought Cartographer and Atlas made some good progress in that direction, but lacked simplicity and ease of use.

Along the way, as features grew, this utility blossomed into a full project oriented utility. It is now capable of handling many different types of text and pointer structures and hopefully handle most needs for your entire project from dialog to item lists. It's the most powerful tool I've ever programmed. The utility uses a project and string oriented approach. Start a new project, add all the files you need. Then add the necessary pointer tables, string blocks, etc. Now you can go to work and translate them right in the utility, or you can dump to full Atlas compliant text output, or even export to XML!

Probably even more important than what the utility currently supports, is the potential of what it can support. With a flexible framework in place and a project oriented interface, the utility can continue to grow and expand to cover many needs. Exciting feature ideas such as integrated script formatting, in-game screen previews, flexible data structure definitions, virtual file systems, and more are possible down the road. Let's not jump too far ahead for. For now, here's what we have so far implemented today!


Images:
http://transcorp.romhacking.net/TextAngel/TextAngel1.png
http://transcorp.romhacking.net/TextAngel/TextAngel2.png
http://transcorp.romhacking.net/TextAngel/TextAngel3.png
http://transcorp.romhacking.net/TextAngel/TextAngel4.png


Features:


  • Batch File Support - Automated dumping across many files.
  • Table Standard 1.0 WIP draft support.
  • Raw Start/Stop Dumps
  • Pointer Table based Dumps (Also can ignore invalid pointers if desired)
  • Pointer List based Dumps for scattered pointers.
  • LOROM00, LOROM80, HIROM, GB, LINEAR modes
  • 8-bit, 16-bit, 24-bit, 32-bit pointer support
  • Little and Big Endian
  • Pointer Spacing (Spacing between Pointers)
  • Pointer per byte spacing (arbitrary Spacing between individual bytes in a pointer)
  • Relative Pointers with Table Start as Base
  • Relative Pointers with Pointer Location as Base.
  • Pointer Global Offset
  • Header support for pointer calculations.
  • AutoDiscovery/Pointer Scan based dumps
  • Atlas pointer compatibility
  • Full Atlas Ready output out of the box. (Currently having issues as this can complicated for some of the complicated features and structures TextAngel supports.)
  • EndTerminated Strings
  • Pascal Strings
  • Fixed Length Strings
  • Fixed Length Strings Based on Pointer Calculations (size=pointer(n+1) - Pointer (n))
  • Fixed Length Lines
  • Dictionary Dump (Uses string prefix/postfix ability)
  • Re-ordering dump by string location
  • Duplicate string detection
  • Overlapping Strings detection
  • XML Export
  • Dump Templates
  • Advanced Tree view with dump preview
  • UTF-8, S-JIS, ASCII, and JP-EUC encoding support for tables and script dump.
  • Project support for dumping all areas of your project. (script, items, menu etc.)

Title: Re: GUI Based Generic Script Dumper
Post by Nightcrawler on Jun 10th, 2010 at 2:48pm
I spent too much time on a decent C# MVP (model-view-presenter) pattern based application framework. It's a struggle to be a good software architect when you're on the learn it yourself program with no formal education or mentor on the matter. I'm still not sure how well the framework will scale, but I gave it a best shot. My error manager and handling framework needs some work for sure, but I'm not really sure how to do it better. Any C# professionals out there with MVP design pattern experience?

I started beefing up my table library to support all the features that have been discussed for the standard. Supports flexible formatting (\n\r\t), end tokens, linked entries, and UTF-8 just fine. :)

I've been been testing it and it's working well so far. It ignores lines it doesn't recognize as supported features and errors out on those that should be supported, but are malformed and unable to be parsed. I thought that was a better approach than spitting out errors on any table that might contain a dump bookmark or other unsupported feature. Both Unix and Windows newlines are supported OK too I think.

I am working on incorporating support for Atlas's address modes (LoROM00, LoROM80, HiROM, GB, Linear).

I'm likely going to have selectable encoding so scripts/tables can be ASCII, S-JIS, UTF-8, or EUC. UTF-8 will be the default.

I've got a host of other potential features I'd like to have for the dumper when I get into it soon. But, I'm going to start small. I'm not sure how much time I want to invest in this.

While it will have Atlas compatible output, I'm trying to have an Inserter built-in as well. Though figuring out the logistics for an equally flexible inserter on par with Atlas is difficult. It may end up with Atlas only output at first and decide if it's worthwhile to develop further.

So, things are moving along. I'd like to get back to progress on Tenshi soon. I haven't been able to touch it since March which spurred my need for better 'universal' approaches and tools. :)

Title: Re: GUI Based Generic Script Dumper
Post by DaMarsMan on Jul 12th, 2010 at 1:31pm
How about an option for pointer scan? Sometimes a script is in order but the pointer table is completely out of order or broken up. Also, can you have an "Add to" box for the pointers? Many systems don't use standard pointers.

Also, the ability to save dump settings as XML would rock.

Title: Re: GUI Based Generic Script Dumper
Post by Nightcrawler on Jul 12th, 2010 at 3:26pm

DaMarsMan wrote on Jul 12th, 2010 at 1:31pm:
How about an option for pointer scan? Sometimes a script is in order but the pointer table is completely out of order or broken up.


I'm not following. What would a pointer scan do? Quick Example?


Quote:
Also, can you have an "Add to" box for the pointers? Many systems don't use standard pointers.


Isn't this the same thing as the Global Offset Box? I could go into more detail with an example. All pointers are added to the global offset. If the pointers are relative, you'd set the global offset to the pointer table start. If the global pointer is 0, the pointers are effectively absolute. Then, there's the option to use the pointer location as the base.

Again, we'd need to discuss an example that you think might not be covered.


Quote:
Also, the ability to save dump settings as XML would rock.


Indeed. Already planned. :) I'd also like to have it so you could point to one of those config files and you could run it from the command line that way as well. Certainly, there will be instances where one might want to run from batch file.

Title: Re: GUI Based Generic Script Dumper
Post by DaMarsMan on Jul 12th, 2010 at 6:14pm

Quote:
I'm not following. What would a pointer scan do? Quick Example?


Not all pointers are within a table. Sometimes they are scattered in the ROM and other times they can even repeat.

<POINTER4> random data size <POINTER2> more data <POINTER1> more more more more <POINTER1>

In this case it's easiest to dump the script as raw and then go back and find the pointers after. However, a pointer scanner may not be totally accurate but it would be able to make some of this task automated.

So a script may look like this


Code:
#W16($0000)
A text string here.<end>

//Pointer not found
Another string here.<end>

Title: Re: GUI Based Generic Script Dumper
Post by Nightcrawler on Jul 13th, 2010 at 11:28am
What algorithm do you have in mind that would scan and be able to find pointers reliably?

The thing is they're not really scattered and the spacing isn't random. I think it would be better to know what you're dealing with. Are they hard coded? Are they in a scripting block?

If so, I'd probably suggest a pointer list for this rather than a faulty scanning mechanism that gives you a guess. Then you can order and rearrange things. I believe Atlas has this ability. I would certainly consider support for an external pointer list.

Now, I've done things where a scripting block is scanned for string commands, or a ROM chunk is searched for opcodes resembling hard coded pointers. In both cases, my results were certainly unreliable and required much manual investigation in false positives or even missed occasions.

What you're suggesting might work better as a separate scanning utility that generates a pointer list to be used for dumping. It could be rule based or have a few different scanning algorithms etc. Maybe even a relative searcher using pointer size and endian searching for increasing values. There's a few things you could do. Though it's kind of like using a corrupter. Yes, it can work, but it's probably better to use techniques that will gives you a concrete and reliable answer.

I have a hard time thinking of an algorithm that would work across different platforms/consoles and pointer size/endian.

Title: Re: GUI Based Generic Script Dumper
Post by Gil Galad on Jul 14th, 2010 at 6:59am
I really think you're going in the right direction with this script dumper. It looks very promising and hopefully you'll be updating it upon occasion.

As far as out of order pointers are concerned, I don't think that will be a problem at all. When you input the range of the pointer table, it simply dumps them to a text file and if you have the script Atlas compatible, the address of the pointer will be written under the small block of text.

I used Cartographer for Star Wars (Namco) and it had some out of order pointers which were handled easily.

One thing about Cartographer that's annoying is that it outputs #W16 for each string that's referenced by a pointer. So, you have to go to each one and change it to whatever label you need. For a really large script, this is a problem and time consuming. I talked to RedComet about adding better support but I don't think he has the time for it currently. I think that Cartographer is a great concept and it works well with a few exceptions.

I think that there should be some type of input box in the dumper somewhere where you can add custom pointer writes. When you dump just output in the text dump file. Instead of doing #W16 for each one, for example.

Here is a small example of how it could look like.


Code:
#SMA("LINEAR")
#HDR($10)
#VAR(dialogue,TABLE)
#ADDTBL("mad_ea.tbl", dialogue)

#VAR(motto, CUSTOMPOINTER)
#CREATEPTR(motto8000, "LINEAR", $-8000, 16)

#ACTIVETBL(dialogue)


//*********************
//**START OF 12842.txt**
//*********************
//GAME NAME: The Most Dangerous Detectives      
//BLOCK #000 NAME:            Dialogue Block (POINTER_RELATIVE)

#JMP($12842,$1300E)

//Displayed - Intro dialog
//POINTER #0 @ $12810 - STRING #0 @ $12842
//タカ  「きのうのおんなはどうした。ユウジ 「ごぜんれいじの チャイムと     どうじに わかれたさ。[SECT]
//タカ  「いいおんなだったじゃねえか。ユウジ 「せいかつを     かえろっていうんだ。[SECT]
//タカ  「おとこを     かえようとするもんさ。     おんなは。[SECT]
//ユウジ 「かえられるか いまさら。タカ  「むりだろうな。[SECT]
//ユウジ 「こんな おもしろいしごと     やめられるか。[END]
#WRITE(motto8000, $12810)


I'm also really tired right now, so if I'm not clear about this, I apologize.

Title: Re: GUI Based Generic Script Dumper
Post by Nightcrawler on Jul 14th, 2010 at 11:01am
What would the idea be there? To reorder the strings for better organization, but have the pointers remain the same?

The issue with the #W16 from Cartographer is what exactly? You have to manually change it to 'whatever label you need'?

Title: Re: GUI Based Generic Script Dumper
Post by Nightcrawler on Aug 2nd, 2010 at 2:05pm
I'm thinking of calling this 'TransText' or 'TextAngel'. I'm not sure yet. Any opinions?

I've been adding more features and testing things out since the last update. I ended up revamping the interface to something that would scale a little better with the new features I was trying to add. I've gone to a tab based interface as you can see here:

http://transcorp.parodius.com/scratchpad/TextAngel.png

I added support for pascal style strings and re-dumped the Herakles IV script to test using Pascal Strings and the new Table File Standard. It's coming out looking nice.

I'm still trying to work on refining/polishing the interface. I'm not even sure how I'm going to handle situations where you want to dump say a directory of binary decompressed files. Or cases where you have a master block of absolute pointers to individual text blocks with their own pointers. Things like that I find difficult to figure out an interface for. I welcome to suggestions.

I've also added a few debugging features that I have yet to make optional.

It's certainly a work in progress still, but it's becoming usable. I'll probably make some more screenshots when I stop changing my mind on things! :)

Title: Re: Text Angel (GUI Based Generic Script Dumper)
Post by Neil on Oct 30th, 2010 at 1:13pm
TextAble?

Although it would be nice if you went with a term that fits with the Cartographer/Atlas schema, especially if you're output is something reinsertable by Atlas. Annotator? Doesn't really have a clever ring to it, but maybe worth consideration.

Title: Re: Text Angel (GUI Generic Script Dumper/Inserter)
Post by Nightcrawler on Apr 7th, 2011 at 1:50pm
Check out the first post for several new images and explanation of how things are currently working. I didn't post here for awhile because I just kept spinning my wheels changing the interface over and over. I'm still not entirely happy with it, but it seems to be the best I can think of. It got especially tricky when I decided I want it to be able to insert what it can dump as well. That's when I thought of a unified interface. You basically need the same information whether you're dumping or inserting, so why can't they share the same interface in the same utility? If nothing else it's good thinking fodder! :)

Title: Re: Text Angel (GUI Generic Script Dumper/Inserter)
Post by Nightcrawler on Jun 27th, 2011 at 11:12am
For those of you checking here for news about this, I was seeking design advice from some guys over at RHDN:
http://www.romhacking.net/forum/index.php/topic,12462.0.html

This gave birth to adding some feature for more advanced usage:
http://transcorp.parodius.com/scratchpad/interesting.png

I am getting there. It's shaping up to be something nice. I will hopefully soon pull together a little write-up on all the features TextAngel supports. Unfortunately, it's still been in a constant state of flux as it morphs and twists it's way into what it needs to be. Aside from being based on an emerging and changing table file standard, my initial design and outline just didn't cover everything. All kinds of new business and missed cases started coming out. Then, my novel idea of dumping and inserting with the same interface became a bit more difficult than I thought as the features supported became more complex. I may be a seasoned ROM hacking veteran, but I still have a long road ahead in good software design. It's been a silly mix. I know precisely what I want and need to do conceptually as far as ROM hacking application. Then I hit a bunch of walls when my program design fails to scale, my user interface comes out awkward, or I'm just not sure how provide for certain ROM hacking cases. Smart guy on the left hand, dumb guy on the right hand, and they have to get together, shake hands, and come up with something good! Sometimes it makes me laugh!  ;D

Like everything I do these days, the pace is slow, but the fruit will be worth it when it's completed. :)

Title: Re: Text Angel (GUI Generic Script Dumper/Inserter)
Post by Next_Gen_Cowboy on Jul 18th, 2011 at 11:24am
It's interesting to see, and I think the basic concept as a whole applies in almost every case. Unexpected problems, a user-base that varies in talent, and skill. Seeing new possible aplications and implementing them, desiging a UI; I think it all just comes with the territory.

The important aspect at least you seem to have covered. You understand what you need, and what it needs to do. Interfaces suck anyway  :P

They're just another thing I could never get right, or make work no matter how I went about it.


Title: Re: Text Angel (GUI Generic Script Dumper/Inserter)
Post by Gil Galad on Jul 26th, 2011 at 7:24am
If Text Angel is at a working state right now, I could possibly test it on a small NES game like I did with Cartographer a couple of years ago. Up to you, of course.


Title: Re: Text Angel (GUI Generic Script Dumper/Inserter)
Post by Nightcrawler on Jul 28th, 2011 at 4:00pm
TextAngel has seen huge progress over the past several weeks! It's in a generally working state and will need much testing. However, it's based on the still not finalized new table standard. As a result, there are several issues with Atlas compatibility with the current version due to some generally incompatible operations, as well as some table standard differences. 

TextAngel strives to not only be Atlas compatible, it strives to be fully 'Atlus Ready'. What I mean by that is it will output all necessary Atlas commands for most of it's script dumps. It will theoretically be ready to re-insert out of the box if you didn't want to make any location changes. It will easily do what Gil Galad  spoke about earlier and use CUSTOMPOINTERS and figure out when using AUTOWRITE is appropriate. I am having some difficulty as many things in TextAngel don't have 1:1 translations in Atlas, and there are incompatibles as TextAngel uses the new table standard and Atlas does not at this time. Also some things are just very difficult to automate Atlas output on, especially when some Atlas quirks are not fully understood by me. ;)

I've also just finished ironing out the AutoDiscovery/Pointer Scan feature where given a text block, it will dump strings and scan/discover pointers in the pointer area based on the specified format similar to what DaMarsMan said earlier in this topic.

I also added a method to do a dictionary dump as well as re-order by string location and duplicate string check this week! I would say it's probably the most advanced public dumper ever.

Here are some of it's many features:


  • Batch File Support - Automated dumping across many files.
  • Table Standard 1.0 WIP draft support.
  • Raw Start/Stop Dumps
  • Pointer Table based Dumps (Also can ignore invalid pointers if desired)
  • Pointer List based Dumps for scattered pointers.
  • LOROM00, LOROM80, HIROM, GB, LINEAR modes
  • 8-bit, 16-bit, 24-bit, 32-bit pointer support
  • Little and Big Endian
  • Pointer Spacing (Spacing between Pointers)
  • Pointer per byte spacing (arbitrary Spacing between individual bytes in a pointer)
  • Relative Pointers with Table Start as Base
  • Relative Pointers with Pointer Location as Base.
  • Pointer Global Offset
  • Header support for pointer calculations.
  • AutoDiscovery/Pointer Scan based dumps
  • Atlas pointer compatibility
  • Full Atlas Ready output out of the box. (Currently having issues as this can complicated for some of the complicated features and structures TextAngel supports.)
  • EndTerminated Strings
  • Pascal Strings
  • Fixed Length Strings
  • Fixed Length Strings Based on Pointer Calculations (size=pointer(n+1) - Pointer (n))
  • Fixed Length Lines
  • Dictionary Dump (Uses string prefix/postfix ability)
  • Re-ordering dump by string location
  • Duplicate string detection
  • Overlapping Strings detection
  • XML Export
  • Dump Templates
  • Advanced Tree view with dump preview
  • UTF-8, S-JIS, ASCII, and JP-EUC encoding support for tables and script dump.
  • Project support for dumping all areas of your project. (script, items, menu etc.)



By the way, TextAngel DOES insert text as well, but it's not nearly as developed in that area yet. It supports all of the same general features for insertion as it does for dumping, but much of the nuts and bolts are not yet implemented there. It does insert the Heracles IV script though. :)

Title: Re: Text Angel (GUI Generic Script Dumper/Inserter)
Post by Gil Galad on Jul 29th, 2011 at 7:57am
Sounds like you've done a lot of work on Text Angel. I'm sure a lot of those functions will be useful to me. I understand many things about Atlas but not all. If you have a question, perhaps I can answer it. I don't know the inner workings, but I know a good deal of the commands used in a Atlas script.

The utility sounds promising though.

Title: Re: Text Angel (GUI Generic Script Dumper/Inserter)
Post by Nightcrawler on Aug 1st, 2011 at 10:51am
I've been speaking to Klarth about it all, so he's clearing up the issues. The #W16 type commands Cartographer uses are actually deprecated and only in there for legacy support of old versions of Atlas. They shouldn't even be used.

Basically, the preferred (per methodology of Atlas) and cleanest method is to use AUTOWRITE with a  PTRTABLE and a CUSTOMPOINTER for all linear pointer tables. AUTOWRITE with PTRLIST and a CUSTOMPOINTER for near all other cases that can be handled with a CUSTOMPOINTER. Outside of that, you'd need to use the individual byte writes WLB, WHB, WBB, WUB for pointers that have arbitrary spacing between bytes such as those in assembly instructions. That covers nearly all conceivable possibilities.

WRITE commands exist only for comfort or need of having those addresses right in the script instead of pointer list. Since many ROM hackers seem to desire this for debugging or wanting less files, I may provide an option for normal and extended Atlas output.

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on Aug 18th, 2011 at 9:07am
I've updated the first post with some screenshots you guys may be interested in. :) Over the past several weeks the GUI has gotten a complete overhaul and gone in a different direction. While the old one was very simple to use, it just couldn't scale to incorporate everything the backend was getting sensibly. So, as the utility has grown and become more project oriented, the GUI needed to become more project centric as well.  I think I'm finally happy with the direction. It's too bad it took me this much work and this long to get there! That's what happens when you don't have clearly defined goal and plan from the start. My original plan was a mere fraction of what I ended up doing. It's really been a good learning experience in software design though. Tearing down and rebuilding until you get it right works wonders. You see what works, what doesn't, reinvent the wheel a bit, and start to understand why some methods and patterns are commonplace today.

I added a translation interface and string search interface, which I think will be very helpful. I find myself always having to look up strings in a project during debugging. Now I can search the entire script and quickly pull the original dump data, translation, and insert data.  8-)


Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on May 1st, 2012 at 11:20am
Now that I got the extremely long side tangent of the table standard mostly out of the way, I've gone back to this.

As you can see from this image, I've now got the program set up revolving around a project tree. How it works right now is you select the file/s to dump from. Then you add  any combination of building blocks you'd like. The building blocks (StringTable, PointerTable, PointerTableTree, PointerTree, AutoDiscoveryTable, PointerList, Pointer, and String) available cover all possible functions my dumper is able to do. Any combination of those items can be added to the files to dump whatever you need in the fashion necessary.

This by itself could be tedious depending on the situation. For instance, my Herc4 example contains decompressed data files 'decompX.bin'. In each one I simply want to dump the contents. I don't want to have to add StringTables to all of those files individually. So, I have a shortcut option 'Add to all Files' so I can add a newly defined building block to all files at once. That solves that. All of the building blocks will generate and dump associated subblocks or strings that may be necessary. So now whether you're dumping from a single ROM file, or various other data files, you can set up and dump sanely so far. It seems OK for the situations I've tried with it.


Insertion:

Due to the fact that I added script editing abilities to it, it needs to be able to do some sort of insertion. By default the hierarchy is set up to insert directly back into the same files and places the data came from. This is not always desirable as you may want to insert many files to a single file, insert to new individual files, or insert to new locations. I thought about changing the hierarchy once again, but it seems to make logical sense to me to just select a bunch of files and tell it what to dump from them rather than create a bunch of blocks (that may all dump from the same file) and tell it the same file all the time.

My current idea is to keep the hierarchy as-is based around dumping and add insertion options on the Insert table on how to insert (All sources insert to single ROM, All sources insert to new files based on pattern naming, Insert back to same source file/s) and what to do with it on the insertion tab (relocate strings or other options I make). This keeps it external to the hierarchy. Otherwise, I can't think of any way to manipulate the hierarchy to better handle the various dump/insert file situations.


Dumping:

If I keep my current hierarchy and internal script editor, and go in the direction I've described above for insertion, I could make similar options on the Dump tab to either (dump to Atlas ready single text file, dump to Atlas ready file per block, dump to XML). These would be the the options in the event you didn't want to translate in-program (in which case you wouldn't be dumping anything outside the program).

On top of that, I'd like to have the ability to import external text files, XML etc from above. This would be useful for scripts that exist already or choosing to use TextAngel to insert after exporting for some reason. It would probably be limited in scope as I don't think I will parse the entire Atlas command set for import and it would make assumptions for some non Atlas format scripts. This is just in the idea phase right now. Perhaps some basic support to start and leave room for import expansion in the future.

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on Sep 7th, 2012 at 11:13am
Since the program has evolved to now center around doing the translation in-program, I've started working out methods to import strings from external files into the project structures. Ultimately, I want to have a variety of import and export capabilities to be able to get things in and out of the program in various formats in addition to simply dumping or inserting. There's always going to be occasions where you might want to say export the script to text, modify it in some other program and import it back into the program or maybe export to XML, Atlas compatible insert file etc.

What prompted me to work on the importing was that I want to import the Herc4 script, which was dumped and translated prior to TextAngel becoming project centric with in-program editor. Having the strings organized editable, and searchable within TextAngel makes management, debugging, and editing far simpler than going through text files. So, there needs to be a way for me to do that.

As always, more flexibility equals more code and more time.

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on Jul 15th, 2014 at 8:44pm
Wow, I didn't realize it had been so long since I updated on this. This project is not dead. I have been updating and expanding it here and there when time allows. It's actually my primary utility for all my current script needs. As such, it's definitely not going away.

I typically work from more than one location via flash drive. Since TextAngel is project based, I decided to make it use relative paths so the whole project becomes portable. That means you basically have a TextAngle project file in the root and all other file references are relative. I think I will end up adding a blend where if a file is outside the main project directory, it will utilize a full path, but I currently don't have a need for that.

I've also improved Atlas compatibility and added flexibility of being able to dump any block node individually, multiple together all in one file, or the whole project split into files or into one. It can be a bit tricky to output compatible Atlas files in all instances.

I added a number of additions to the native insertion abilities as well. TextAngel can do a number of things Atlas can't since it has an break down of the entire text structure right there in the dump. The potential is huge there, but I haven't gotten all that far yet.

In the more immediate future, I'd like to get some command line usage working for when you don't really need the GUI anymore  and just need to build it via batch file and what not. Oh, and one of these days I've got to finish implementing the full table file spec that I drafted. I would agree with some of the critics that it is quite complex. All of my constituents insisted these things be included, so it drags on. I don't really need some of that functionality for my current projects, so I've had little motivation to work on the more complex aspects.

Unfortunately, as afar as a public release goes, I bit off more than my time can chew on this. There's still so many incomplete features, no documentation, and bugs requiring workarounds. I wouldn't expect any major developments here for awhile yet. Although, lately I have been starting to catch up on all my backlogged tasks relating to TransCorp so you never know. ;)

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by breech on Jan 5th, 2015 at 7:52pm
I signed up to give you a nudge  ;) I really hope this project doesn't fall off the radar. Keep up the good work!

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on Jan 18th, 2015 at 6:36pm
Concentration the past few months has been on moving Heracles IV toward completion and the recent maintenance releases for Dual Orb 2, Wozz, and Emerald Dragon. So, I haven't really done anything new with Text Angel recently. Heracles doesn't require much further feature development, so I may not get back to this until I move back into Tenshi No Uta work.

More recently, I was sick for awhile in December and then again soon after new year. Couple that with some tough times at work and a month goes by without doing anything significant ROM hacking wise. That's the way the cookie crumbles sometimes though. I have also been taking some time to play through some more games. I spent too many years doing too much ROM hacking related activities and not enough playing. Trying to strike up a better balance.

So, there's your impromptu mini update!

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Garrett on Jan 20th, 2015 at 6:25pm
Perhaps if you can finish most of the hacking on your translations in the next year or two, you can get involved in some emulation dev? The state of SNES debugging tools is really quite pitiful when compared to those available for the NES. Furthermore, development of SNES debugging has been stagnant for several years.

Plus there are other systems with active emulator development with a focus on debugging tools and meanwhile the SNES is stuck in an older generation of suck.

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on Jan 24th, 2015 at 5:36pm
I tried to procure the source from Geiger for his SNES9x debugger a year or two ago. He wouldn't give it up. I would love to update that to the newest version, fix some of the problems with it, and add a few new features. But alas, no can do.

I have no interest in working on the higan source to add a debugger for a number of reasons, so I won't be doing that.

I can't think of any other open source SNES emulators that would be worthwhile.

I was hoping Martin would do something better with the debugger in no$sns being the newest SNES debugger. It's got some interesting concepts, but it's a little odd in design. Have you used that one before?

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Garrett on Jan 26th, 2015 at 5:56pm
I don't like any of the no cash emulators. Although I only recently used no$gb and I had issues running it on my computer.

Edit:

Here we go, now this has promise.

https://github.com/devinacker/bsnes-plus

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on Jan 31st, 2015 at 4:28pm
Looks interesting. No binaries available yet that I see. I have no interest in setting my machine up to compile everybody and their uncle's project to try them. That's why I don't use Linux. :P

Title: Re: Project - Text Angel (GUI Generic Dumper/Inserter)
Post by Nightcrawler on Oct 13th, 2015 at 1:18pm
Steady progress has been made on TextAngel for quite some time and the dumping side is quite advanced. I'm becoming pretty happy with it. However, I have been caught in viscous design circle because of no suitable insertion option. I still can't finalize the dumping side without having an understanding of the insertion side, which has changed several times now.

Originally, my initial scope was going to have TextAngel be a dumper and use Atlas or external utility to insert. This ultimately was not a good enough insertion option for me, so I wanted to add native insertion abilities to TextAngel. It's here that I have hit a number of design hurdles that I can't figure out a sane way of handling.
Dumping is pretty good. You have a number of different types of nodes that will dump your strings and maintain all of the hierarchy structure. You can then translate all the strings in-program, format, dump the strings to files etc.

With all of the structure there, everything is set up and simple to insert exactly back to where it came from. But, it's rare that you'd actually want to ever do that. You can't fit translated text back where it came from. You aren't going to want to use the same table files used to dump. Likely, you'll want to relocate everything too. Maybe you'll want to insert to the ROM, or maybe you'll want to insert to individual files for further external processing such as compression. Furthermore, you often times want to change the structure upon insertion. This is where I have a huge disconnect...

How do I account for such tasks and maintain the translated strings? At first I thought I'd simply have a few insertion options you could configure upon telling it to insert to cover relocations or table changes. However, it really became very inflexible and too limited that way. Next, I thought about a second insertion view where you can build nodes to create an insertion structure. However, I couldn't figure out how to properly link up and maintain the strings/translations you have from the dumping stage. The structure could be entirely different. You could change fixed length items to pointer table based, or you could take 16-bit dialog pointers and make them larger, or you might do any number of anything affecting structure. The only thing I could think of was a painstaking process of exporting the strings from the dump structure and importing them into the insert structure if different. However, that is prone to error when they are not easily compatible formats, and difficult to automate for the user, especially if many files or text blocks are involved.

I am having a tough time figuring out where to go from here. It's stalled out the dumping end of things because I can't finalize the dumping structure without knowing what the insertion solution is.

Nightcrawler's Message Board » Powered by YaBB 2.5 AE!
YaBB Forum Software © 2000-2010. All Rights Reserved.