FreeCAD BCF-Plugin development blog

Blog to the development of the BCF FreeCAD plugin.

Dev Logs

This is a daily updated log of the work I do on the BCF-plugin for FreeCAD

August 19th: Wiki update! Yorik made me aware of an import error persisting in my code, as well as of the fact that there is a property for QTableViews to automatically let the last row/column stretch till the end of the widgets heigth/width. Both things were fixed/integrated in commit f74b901 directly on the master branch.

I also tried to recreate the misbehaviour, Yorik was experiencing, concerning selected comments not being readable. Thus I installed a dark theme myself in hope of reproducing the issue but for me it worked fine.

Other than that I updated the wiki pages to more reflect the current state of the repository.

August 18th: No work done today.

August 17th: No work done today.

August 16th: Documentation rules! Apart from documenting code in commit 98542fd, I added debug logs to writer.py, reader.py and the programmaticInterface.py. This was done in two commits:

commit def313a adds placeholders to the topic metrics window's line edit fields, where it is unclear what value has to be entered.

And then finally the final release for GSoC'19 was released, v1.0.

August 15th: Today was mostly about documentation. Although, I changed the behavior of the notification label in the topic metrics window a bit, as well as fixed the bug that prevented the label from showing. The added and deleted lines comprise the commit e8a116b.

Then commit 4a72793 documents the classes RelatedTopicsModel and AdditionalDocumentsModel.

commit 96e85f1 fixes the bug where a missing guid in the document reference type is read as None value instead of getting assigned its default value.

commit 5141a27 adds documentation to the complete bcfplugin.rdwr module. Although files are currently not documented. This will follow tomorrow.

On the wiki gui tutorial page I added two sections. One showing how to create a complete new project, the other one showing how to add a topic to an open project.

August 14th: Working towards the final release for the GSoC'19. In that sense I added some documentation in the "then" plugin_model.py and plugin_view.py files. The associated commit is commit 0ae7746.

commit 94a6f04 completely restructures the code of the gui subfolder. Now every view, delegate and model has its own file. This is still on the develop branch since I am not quite confident that I didn't introduce some bugs with it.

commit 49915e2 adds the LGPL-2.1 license preamble, which is used by FreeCAD also, to every source file. Also the license itself is added to the project in this commit.

Apart from that I added a section to the gui tutorial on the wiki page, explaining the topic metrics window.

August 13th: The third release is out! But before that was the case some changes were made:

  • commit 38529f5: Removes the margins from the outside borders of the layouts to use more of the available screen real estate.
  • commit 4cdbecb: Lets the plugin add the *.bcf and *.bcfzip file types to the list of supported filetypes of FreeCAD and registers a handler module for opening such files.
  • commit 5bc6675: Pushes the release v0.3.

Apart from this, I updated the README.md and the Tutorial/GUI page on the wiki to reflect the current release. I documented most functions in ./bcfplugin/gui/plugin_view.py. And I moved to QSplitter in the plugin. Thus now the plugin is comprised of roughly three sections which are all resizable by the QSplitter.

August 12th: Dark theme should now be supported out of the box! Since today I have written rather many and small commits I will just list them and write a short description to each one:

  • commit a770ba4: adds a function that calculates the minimum vertical size of tables and sets it. It is used in the topic metrics window
  • commit 827849c: fixes the issues with the dark theme. The solution here was to not use hardcoded colors for the comment list, now the QPalette is utilized.
  • commit b0df2f1: moves the delete button a bit downwards in relation to the start of a comment.
  • commit 1650261: replaces the topic combobox with a list of topics.
  • commit 16733bd: Fixes a bug in the topic model, stemming from the time where a combobox was displayed instead of the current list. If the first topic would have been double clicked, nothing would have opened. This off by one error was located in the model itself.
  • commit 8a6638e: removes margins from the comment and snapshotarea groups. Now more space is used for actual content of these two groups.
  • commit 5f9c48c: changes the flow of the snapshot list from (default) vertical to horizontal.
  • commit 50d3fc1: adds a bcf file designed for testing the snapshot list, whether it rightfully scales its contents.
  • commit 2dcf7a0: documents functions in bcfplugin/__init__.py and bcfplugin/programmaticInterface.py, also it deletes some unneded global variables from bcfplugin/__init__.py and bcfplugin/util.py.
  • commit 44f6157: merges the feature branch feature/create_project into the develop branch, and thus adding the feature of creating whole new projects from inside the plugin itself.

August 11th: No work done

August 10th: No work done

August 9th: commit b0bb316 adds a filter to the log messages printed to the report view of FreeCAD and stdout in the way that validation errors reported by xmlschema are only written to file.

commit bc4062c adds a stretch beneath the project group to force the plugin to assume its full size right from the start. After a project is opened the stretch is deleted again.

commit 8551955 enables the user to also add topics to an existing, open project. Details are gathered through a separate dialog with a form layout.

commit 1e7c87c changes the default insertion index of new elements to denote the last possible position in a node. This prevents errors where an element is inserted _before_ its predecessor. More details are written down in the commit message.

Currently I am working on a feature to also add complete new projects. But this is still done locally.

August 8th: More stable logging now available! Today the code was adapted to only use python's logging library. To integrate this also into FreeCAD a custom logging.Handler was added also. Everything now is logged to bcfplugin_log.txt in the working directory, which will be created already in bcfplugin/__init__.py. The corresponding commit is commit 290e805.

commit 2f9e20f just adds a little notification message when either an element in the first or second column of the additional documents table in the topic metrics window is selected. The notification informs the user about the effect a double click on that element has.

The other commit for the day is on the feature branch feature/topic_list, commit 64d1e76. It rearranges some UI elements in the top half of the plugin to encompass a ever visible list of available topics. This has the advantage to the combobox, that one click less is needed to open another topic, which might decrease friction.

August 7th: Today usability improved by a bit. In the topic metrics window the list showing additional document references was enhanced. If an element in the "Description" column is double clicked, and it is coloured in blue, it is opened with the standard application for that file type on the system. If on the other hand the path could not be resolved, the element in the second column can be double clicked, resulting in the path being copied to the clipboard. The associated commit is commit 69ec3b7.

The other commits for the day are mostly autogenerated by gitkraken, as I also released the second release version of the plugin today on the master. The release notes can be found in commit 86d0c49.

Apart from this I am working on moving my self implemented logging "framework" to the one in python's standard library: logging As this is still in early development, I am working on it locally. Tomorrow I will have switched the plugin fully.

August 6th: Let's start right away: commit 7c847e4 fixes a, or rather adds a workaround, to the erroneous behavior occuring when the user wants to add a comment before a topic is selected. Prior to the first selected topic the line edit for a new comment is simply disabled. Additionally this commit adds a window title to the plugin.

commit 2413291 writes the errors reported by xmlschema to a error log file contained also in the temporary directory. Before the first line is written however the user is notified about it and given the path to the log file. This mechanism uses pythons logging library.

commit f72fb9a adds a prefix to all temporary files and directories created by the plugin. In addition to that, the plugin deletes all files from the temporary directory with that prefix to rule out artifacts from a previous session.

commit 27ca9d9 adds the pI.closeProject() function. It is solely intended for the nonGUI mode and sports some interactivity in the case of a dirty state.

Then as per suggestion of @saso from the FreeCAD forum, I swapped the combobox for the topic selection out in favor of a list showing all available topics. Work on this feature is currently done on the feature branch feature/topic_list. I posted a little demo on the FreeCAD forum, you can see it here

August 5th: Today was comprised mostly of bughunts and their fixing. It is staggering how much time just simple bugs can consume to fix.

So one that got to my nervers in recent days was an error message constantly being displayed right after a file was selected for opening. This was fixed in commit 78d0724.

Another thing, I already noted yesterday, was that the save before exit dialog still showed up every time the plugin should have been closed. This is now also fixed by introducing a "dirty bit" in commit 5bf328f. Also this commit morphes all imports of each source file into absolute imports, only affecting plugin local imports.

commit 65aae9d adds metadata to the BCFPlugin.FCMacro file, after I read about how to integrate addons or macros into the official FreeCAD repositories.

commit 1167fd8 adds a check for None inside the ModifiedDate.__str__() function. This lead to an error if an original comment was being modified that didn't have any modified date set prior.

The wiki page got extended by one page, talking about the GUI, the concepts implemented, the UI elements and their behavior and the design decisions during development.

August 4th: For most of the time mostly I was on a bug hunt which resulted in not storing the path to the temporary working directory in a tempfile.TemporaryDirectory object, rather I store the path to it in a file in the temporary directory of the system. This switch was done in commit f8b0fc9.

Then the second commit for the day, commit ee46345 implements the "close-without-saving" dialog's behavior how I like it. Previously the dialog did not close after the file was saved and thus prevented the whole plugin from closing.

August 3rd: Today not that much code was put out.

commit 5565b03 adds a little behavior change to the topic metrics window. Since there may well be some topics that don't have any additional documents listed or are linked to any other topics, the corresponding lists are left hidden. However, the groups containing the actual lists still persist and subtly indicate that this aspect of a topic does not exist in the BCF file.

commit 185fe50 is an attempt towards internationalization. Here all GUI strings, descriptive ones, not the ones displayed as content of the BCF file, are passed to the translate() function of QtWidgets. However, no other languages are currently supported appart from english.

commit d25fdd1 moves the prompt for the user's email address away from plugin_delegate.py into plugin_model.py. The reason behind it was that apparently the code inside delegates is executed in a separate application context. The erroneous behavior that led me to that conclusion was that every time the user's email dialog box showed up, a new temporary directory was created for the author.txt file, which holds just the email address. This is strange because for one: at that point in time a temporary directory is already created and for the other the reference to the TemporaryDirectory object is stored in the plugin global variable TMPDIR declared inside bcfplugin/__init__.py. Thus a new temporary directory will only be created if TMPDIR equals None. Now TMPDIR gets set right when a new BCF file is about to be opened, into it the contents of the file will be extracted. A simple test (just prints of the the TMPDIR variable from plugin_delegate.py and plugin_model.py) showed that in former it referenced None while the latter referenced the "correct" object.

Apart from these three commits, I started writing on a blog post about my work done in during the last three months. It shall serve as main document for the project submission. As soon as it is finished I will start a discussion about it in the forum.

August 2nd: commit 9519986 fixes the 'Reset" button that resets FreeCAD's view to the previous setting. Before it also was shown even when the view settings of the viewpoint could not be applied. A following press of the button resulted in an exception.

commit 14e6282 fixes the stack switcher and the stack widget in the snapshot group. Before both were not reset to the first item/widget on a switch of the topic. Now they are.

commit 9b443c7 corrects the deepcopy algorithm for an object of project.SimpleList. Previously every copy encapsulated the elements (which are of type SimpleElement) anew in a SimpleElement, thus creating something like: SimpleList -> SimpleElement -> SimpleElement -> Value.

commit 9828fa1 adds a list view to the topic metrics window displaying all related topics specified for a topic.

commit efca76c adds support for comments without text. Previously they could be read in, but when the topic should be displayed, containing such comment, an exception was raised resulting in no comment shown at all.

commit b8baebe fixes a bug where elements could not be modified because the containingObject member has been overwritten with the contents of the supplied copy, which was None.

commit 429bb70 restructures code inside the plugin_*.py files and in the programmaticInterface.py file. For information on how the code is structured now, please refer to the commit message.

July 31st & August 1st: no work done.

July 30th: More and more is possible! Today I implemented the desired behavior for the modification of modified comments and topics in the case the user wants to stay private and does not enter his or her email addres. For more info please see the accompanying forum posts. The corresponding commit, however is commit 2b7f68c.

commit 0601f82 is a small one, it just fixes the datetime format with which the creation/modification date of a comment/topic is displayed to the user.

commit e50f7ed moves the tempDir variable from util.py into bcfplugin/__init__.py in an attempt to a bug where two temporary directories are created in subsequent calls of util.getSystemTmp(). The first one is created as working directory, where the bcf file gets extracted to and so on. The second one is, wrongly, created just for the file containing the author email address. Somehow between these calls the state of util gets lost. Maybe it has to do something with Qt. We will see.

commit a5696bd adds the capability to viewController to reset the view of FreeCAD to the previous state, it had before a viewpiont from the BCF file was activated.

commit d494214 adds a button to the plugin that triggers the reset of the FreeCAD's view. This button is only shown if a viewpoint has been applied and gets hidden again after the view was reset.

By checking out commit eb8f4ca you will be able to activate a viewpoint through the plugin by just double clicking the desired viewpoint element in the viewpoints list.

Last but not least I added a new page to the wiki, detailing about what personal data can be stored in a BCF file, and what is/is not collected by the plugin.

July 29th: The packaging work has begun! On this day I fixed some bugs on the master branch. The bugs were:

  • "list index out of range" exception during window resizing
  • "None typ ehas no attribute referencedViewpoint" when selecting special comments
  • not really a bug, but a Deprecation Warning from Qt.

These fixes can be found in commit commit 9e4382b.

A big thing that happened today was the merging of the feature/custom_deepcopy branch. No the plugin should be a bit more responsive. It is currently on the develop branch. The corresponding commit is commit 073d078.

commit 3f0ce7f on the other hand does not add new functionality, it just fixes an error that occured during merging everything into master yesterday. In particular the function writer.createBcfFile was renamed writer.zipToBcfFile. These changes have been applied to the pI also in this commit.

Locally I am currently working on the optional email thing. The approach I am taking is that either the user enters a valid email address or leaves the line edit field blank. What is still missing? I am planning to also delete the modified author in the case where the user modifies something but didn't enter an email address. This shall prevent masquerading of changes.

The second thing I am working on locally is a nice integration into FreeCAD. Currently the state is that it can be opened in the task panel and is fully functional, at least at for now. It still has to be tested.

July 28th: Today I did no work.

July 27th: The plugin is now usable! Today I added the functionality, to the UI, that before the user can do any change to the state of the project, he/she is asked for his/her email address. This is just done once per session. The accompanying commit is commit 4f06257.

commit 46f0502 adds a date validator to the DueDate row in the topic metrics window.

In addition to that, I pushed this state onto master as my first release with version 0.1. Though note that this version is still not tested inside FreeCAD itself. But yeah. I am quite confident now in the plugin. Over the next period I will fix some minor issues and do some packaging for the plugin.

July 26th: More development work today than the last couple of days. First of all I finished the clipping plane stuff. Clipping planes can now be created, commit 0d20165 is the one to go if you are interested in how it is realized.

commit 6d4727d adds stuff, but that was not done today. I noticed that I still had a git-stash lying around with useful changes. For one the colour regular expression gets fixed in this commit, some informative prints are added concerning the colouring of components and viewpoints will be coloured in pI.activateViewpoint().

commit d28b044 adds a nifty little feature linking viewpoints and comments. As you may know, a comment can reference a viewpoint, as you also may know the plugin sports a list showing all available viewpoints. Now, when the user selects a comment that references a viewpoint, the viewpoints list will be automatically shown and the referenced viewpoint is selected.

The main part of my work today, however, is still offline and concerns a separate dialog window that shows the data available on a topic, and also makes it availabe (the parts that I consider mutable) for changes by the user. But it still is not finished as I have to add a Delegate. The Delegate is needed because the user might change the dueDate which has some restrictions on the set of possible values that might be entered. This will be done with a delegate in the end.

July 25th: Today I checked that my algorithm for drawing lines worked, and thus have written a simple test case (viewController-test.py) that draws the lines, specified in a prepared BCF file. These three lines are 5000mm long and lie directly on each coordinate axis. This was done in commit 1910afd.

Then I have written a nicer introduction to the wiki pages, including a short BCF primer, to get anyone started on this topic.

The remaining time of the day I investigated how clipping planes can be drawn in FreeCAD. Therefore I first looked into the source code which led me to the documentation of the coin library. In addition to the knowledge of how to use it I also wanted to know more about the mathematics behind it. At the end of the workday I felt confident to start working on the function createClippingPlane() in viewController.py, which is for one not finished and for the other not pushed to github. That I will do tomorrow.

July 24th: Right now, during closing FreeCAD for the day, I thought "Man I like FreeCAD more and more!"

Now, after this insight on to the dev-log for this day. I finished my update of the wiki pages, and the README.md file. It all is still located on the feature/gui branch. These two aspects of the wiki update are done in two separate commits. commit ab78a0f adds a new section to the README.md file, dedicated to the Qt user interface. commit c8705dd then adds a new page to the wiki. It gives an introduction to the graphical user interface of the plugin, and explains all aspects of it (that were implemented till today).

After this I made another attempt to applying viewpoints, because I didn't believe that my implementation was correct. Two major insights were the result of this effort:

  1. The underlying unit of FreeCAD (on my system at least) is millimeter, not meter.
  2. The field of view is to be set in radians and not in full degrees.

These two insights were then molded into code in commit 47c1151.

Since a BCF file can also specify lines to be drawn, as well as whole planes. The first step would be to just draw lines and so I did. This resulted in additional code (two new functions and some new state variables) in viewController.py. The corresponding commit is commit 64d381c. The new functions are

  • drawLine(): draws one single line using the draft workbench and returns it.
  • createLines(): takes on the lines object from the viewpoint inside the data model and tries to draw all lines. If one cannot be drawn it is simply skipped.

July 23rd: Today I have updated the wiki page of the project, as well as written a small wrap up of the stuff that was added to the plugin since the last evaluation period.

July 22nd: I fixed some issues I had with the deepcopy implementation. Thereby I focused on the modification of a comment. Here it was the case that for one the states of objects were not copied, which prevented the writer module from updating anything. This part got fixed in commit f73b8b1. Then commit 975ba91 fixes the issue of not properly updating the ModificationAuthor and ModificationDate fields inside a comment.

commit 042859c merges my feature development branch feature/gui_switch_to_relative_sizes into the main gui feature branch. The relative sizes are now calculated using the QScreen object of the screen the Qt application is running on and with it calculating the pixels per millimeter. That value is used to convert the distances (given in millimeters) into pixels.

July 21st: Also no work done today.

July 20th: Did not work today.

July 19th: Today I spent my time just on the deepcopy topic. As it turnes out it is not as easy as I thought to create correct deep copies of objects in my data model.

Things to consider for creating a deep copy:

  • The unique id, created in the constructor of every class in the data model has to be copied to stay exactly the same. Otherwise the search algorithm implemented in project.py does not work anymore. The plugin has to be able to search for an object in the original data model and in the copied one by the same unique id.
  • The state of an object has to be copied also, otherwise the writer module won't make an update or in the worst case, delete an object which rather should be modified.

But a few words on how I am implementing the custom deep copy:

Since it is not best to copy the whole project, when just copying a single comment, somehow the hierarchy of an object (Hierarchy.containingObject) has to be ignored. But if the member Hierarchy.containingObject does not get set in any copy function, then the writer module does not work anymore. Reason being that for every new update a deepcopy of the project and the modified element is made. During an update however the hierarchy of the modified element is required, which cannot be created anymore, since Hierarchy.containingObject did not get set in the copy process. To solve this issue, I decided to copy everything downwards the Hierarchy. If for example a copy of a Markup object shall be created then everything referenced by that markup object is also copied, but the containingObject of this Markup object is not copied. The actual way that this can be accomplished is really simple: each __deepcopy__() function has to set the containingObject member of its copied members after the copy was being created. To illustrate it consider the following code:

from copy import deepcopy

class Markup(Hierarchy):

  def __deepcopy__(self, memo):
    ...
    cpyid = deepcopy(self.id, memo)
    cpyComment = deepcopy(self.comment, memo)
    cpy = Markup(...)
    cpy.comment = cpyComment
    cpy.comment.containingObject = cpy
    cpy.id = cpyid
    ...

Here you see for one that containingObject is set, and also that the id member is copied and overwritten in the new object of Markup. This way it is guaranteed that a copy from Markup is only copying everything below it (i.e.: every member of markup), but the containingObject of Markup itself is left untouched.

Since this deepcopy topic is still really buggy, I don't have any commits to show, all work is still done locally.

July 18th: Not much dev work done today, although I have written quite a number of lines. Aside from switching to relative distances in the UI, I also make an effort to increase performance of the plugin a bit. Currently it is the case that for every inquiry of the programmatic interface, if information shall be retrieved (like a list of all comments), then in the process of making a deep copy of every comment, inevitably a deep copy of the whole project is made. Why? Because of the Hierarchy interface, which provides each implementing class with a reference to the class that holds the reference to it. That means, during a deep copy operation, python will stumble on the reference to the containing object and make a copy of it too. But the containing object again has a reference to its containing object and so on. So in effect, if a copy shall be created for the modification date, the complete project with all its topics is copied too, which is a huge overhead! To solve this performance problem I implemented the special function __deepcopy__() into every class that inherits from Hierarchy. But the test cases don't work anylonger with these changes. That is were I left off today.

Before implementing __deepcopy__() I made an effort to understand rotations in the three dimensional space, using Euler-Angles (yaw-pitch-roll), a rotation matrix and quarternions (which are really cool btw.). Also I looked into how one can be transformed into the other. And I got stuck at trying to recreate the example given on the wiki page, unfortunately to no avail.

July 17th: Now two options are available for exploring the available viewpoints. The one was already added and is the SnapshotBar. It is still lacking the functionality to activate a viewpoint when a certain click event happens. The second option is a viewpoints list. It lists all the viewpoints available in a selected topic. If this viewpoint also references a snapshot file then an icon of this snapshot file is shown beside the viewpoint filename. The icon can be changed in size by calling ViewpointsListModel.setIconSize(), per default it is set to be 10 millimeters in width and height. Sizes are expected to be given in millimeters. The commit adding the Viewpoints list is commit 18c5b9e.

The next commit, commit 24bd929, somehow is separate from the above one, but is also required by it. It adds the function util.getCurrentQScreen() to util.py. This function returns the QScreen associated with the screen the current Qt application is shown on. This serves the purpose of retrieving the correct DPI setting to be able to convert the millimeter sizes to pixels.

Then the bug fix, which cost me some time (3.5 hours to be exact), of commit fe7d195 fixes the issue where the size of a comment list element is not properly resized, as its painted area increases or decreases in height.

The remaining time today, 3 hours, I spent with applying the viewpoint settings to the active view in FreeCAD. As many of you will know, Quarternions are a great tool for representing rotations in the three dimensional space. When applying the camera settings, inevitably a rotation has to take place, which are done in FreeCAD using ... exactly! Quarternions. Initially I thought that I didn't have to be concerned with Quarternions as some library functions exist that handle it for me. But during the application of the viewpoint settings, I noticed that my approach to rotating the camera does probably not work. So I walked through this explorative video series, which is great, to get a better understanding of them and how to operate with them. I want to better understand the 3D rotations and how FreeCAD does them, to be able to correctly rotate the camera.

July 16th: I finished the SnapshotBar I was talking about yesterday. Till now it just shows a maximum of three snapshots. The way I implemented it is a bit hacky because I didn't find a suitable way to just display labels, that contain a pixmap, inside of a list view. So I am just using the Qt.DecorationRole to display the icons of the list elements, and nothing else. To check it out see commit 0938ac0.

In the second "major" commit today I implemented word-wrapping for the comment text. For this I integrated the width of the widget into the associated delegate class (CommentDelegate), I couldn't find any other way to access, reliably, the current width of the widget, which serves as base for the wrapping calculation. This is still in development and is buggy. For example: the list items do not properly resize when the width of the window decreases and the comments are wrapped. But on the upside: the bug with the wrongly placed Delete button was fixed during developing the comment wrapping. All of this is contained in commit d20a3a9.

July 15th: Today I mostly worked offline, and only published one commit. The commit 3642e47 adds the feature that the color of a comment is blue if it references a viewpoint. Otherwise it will be drawn in black.

Most of the work today, I did on the "SnapshotBar", as I call it. This shall be a collection of classes (comprised of model classes and view classes) that display small versions of the snapshots contained in the project. It shall present three snapshots in a row, and if more snapshots are present in the topic then a vertical scrollbar is available.

However, I am currently still struggling with getting the images to show using a QListView. I tried returning a QLabel from SnapshotModel.data() into which already the desired picture is loaded, but it did not work that way. Hopefully I get it done by tomorrow.

July 14th: Today also I did not work on the plugin.

July 13th: I did not work on the plugin today.

July 12th: The usability of the plugin was greatly improved today!

Most of the time today I was working on the feature to delete comments from the UI. The way I want to do it is with a button that appears on the right side of the comment when the mouse hovers the comment. It still is not perfect, but already usable. The accompanying commit is commit 051622c.

commit 051622c contains one particular line that I changed. It was a higgs-bugson, at least that is the most fitting classification. The behaviour expressed was that comments were deleted by pressing the button... in the file but not in the model. Strangely my testbench for deleting objects, especially comments, worked. After long debugging I noticed that my policy of not exposing the real working data to the UI came back to haunt me. My pI.deleteObject function looked like this:

def deleteObject(object):
  global curProject
  realObject = searchObject(object)
  realObject.state = State.State.DELETED
  writer.addProjectUpdate(curProject, realObject, None)
  writer.processProjectUpdates()
  curProject.deleteObject(object)

The last line here was the culprit. It is responsible for deleting the object from the data model after it was deleted from the file. Here I used the wrong reference, namely the one of the copy of the real object.

Then commit b156671 adds a save button, that opes a "save-file-dialog" and lets the user save the current state of the working directory.

commit 253e3a9 fixes the bug where the comment list was not reset when the topics were switched.

There are still some commits I pushed today, but these were the most notable ones.

July 11th: Qt is easy to start with, but hard to get right.

I today was mostly on bug hunts, why some stuff was not showing or behaving as I wanted it to. Like for example the horizontal scrolling in the comment view. It somehow did not draw the correct contents when scrolled horizontally. Furthermore was the horizontal scrollbar kind of inconsistent. It did not always show up. Both these issues are fixed in commit f82e40a. The first bug was resolved by correctly setting the drawing position according to QStyleOptionViewItem options. The second bug was caused by an incorrect calculation of the length of an item in the list in plugin_delegate.sizeHint().

What is particular noteworthy is that the plugin now can also be opened in the taskpanel of FreeCAD. This functionality was added in commit b2ebca5 and can be used (will be streamlined in the future) by executing the following two commands inside of the FreeCAD python console.

import bcfplugin.gui.plugin_panel as panel
panel.launch_ui()

I also had an issue with FreeCAD itself and its behavior of swallowing exceptions that are thrown inside of my plugin. It is really nice that an exception inside an outside plugin does not crash FreeCAD, and that is how it is supposed to be. But an error message, that an exception was thrown inside the plugin, would sometimes greatly improve debugging. The bug that made me aware of this fact was that in the comment view comments were shown in FreeCAD on my PC running ArchLinux but were not shown on the virtual machine running Ubuntu 18.04. This behavior was rooted in a missing import of QtCore.Qt in plugin_delegate.py and plugin_mode.py. Strangely, though, it worked on my PC when it actually shouldn't. Anyways, the fix is contained in commit 9cfb5fa.

Finally, not only bugs were fixed today, also something new was added to the UI. Below the comment list a QLineEdit is now accessible for adding new comments. A new comment can be submitted by hitting enter after finished. Not only the comment has to be entered into this field, however, the author's E-Mail has to be appended to the comment, separating the two with ' -- '. If an invalid comment is about to be inserted a tooltip will be shown with a guide to how the text shall be structured in order for it to be added. This functionality was added in commit 47eaded.

July 10th: My work today boils down to this:

  • The comment list is finished, apart from a small bug when scrolling horizontally
  • The programmaticInterface got a new function to apply visibility settings of a viewpoint to the objects in the view.

So now the more detailed version:

commit fc93660 fixed two bugs when painting the comment list. Both were caused by a wrong use of Qt. Previously the position, at which the next comment should be drawn, was calculated by hand. Now this position is taken from the argument options and its member rect.

def paint(self, painter, option, index):
  topY = option.rect.y()

commit c93b004 extended pI.getTopic() with some context awareness. The general approach is to not expose the data model to the UI layer. Thus for every retrieve action, requested from pI, a deep copy of the actual object is returned instead of just the reference. pI.getTopic() however is used inside and outside pI. If it is called from inside of the same module the correct reference to the actual element shall be returned, if however called from the outside a copy has to be created and returned to the calling function. pI.getTopic() is now able to do this, using the inspect module.

commit a702021 integrated the pI into the model of the comment list, this commit therefore made it possible to view actual comments of a bcf file that gets opened during runtime.

commit 894de41 introduces the logical next step to the previous commit. It integrated the comment list into the existing plugin, which previously could open a project and let the user choose between topics. Now, after the user has chosen a topic, all comments will be visible and available for modification. The modification however is constrained with a QValidator.

commit 9814bb4 adds the functionality of displaying a small pop up window showing an error to the user.

commit bc96642 contains the functionality of applying visibility settings to the objects in the currently open view.

To checkout the current state of the plugin run the following command from the directory ./bcfplugin/gui:

python plugin_view.py

July 9th: Well I have learned a lot about Qt and how I can customize existing views with delegates and models. That said the main advancement of today was the creation of the comments list, how I would like it.

For this development of the comment list I have opened a new feature branch ontop of feature/gui called feature/gui_comment_list. commit 5f242fd adds the first (usable) version of the list. It is based on the model/view approach of qt and uses a custom delegate to display the list items. The development files are located inside of ./bcfplugin/gui/comment-list/. To try it just run

python mainwindow.py

from inside the before mentioned directory.

July 8th: Today I started with the first version of the gui. It is completely contained in ./bcfplugin/gui/plugin_view.py, but uses ./bcfplugin/gui/plugin_model.py to get the data to display. Currently when plugin_view.py is run the user is given the option to open a BCF file, through an QFileDialog. If one was selected the gui removes the "open-file-section" and replaces it with:

  1. a label displaying the project name
  2. a label just displaying "Topic" and a combobox filled with a list of the available topics.

The commit adding the two files is commit 6887d52. commit 85d1e8b finishes function viewController.colourComponents() that applies the colour specified in viewpoint.bcf to the (also in viewpoint.bcf) specified components.

July 7th: The weekend I did no work for the plugin

July 6th: I didn't do any work today.

July 5th: The first steps to the gui part of the plugin are made! But first things first. As the programmatic interface is nearly finished in its basic functionality, I merged the feature branch feature/PI_retrieval into develop. This is done in commit 230c1d5.

commit 0a27fd2 adds the functionality to writer.py to add a project file and create a new bcf file. A new BCF file will at first only exist in the temporary directory until the function writer.zipToBcfFile() is called.

Now onto the gui stuff: I added a new branch feature/gui on which I will develop the gui part of the plugin at first. On this branch already some commits exist but the most notable ones are: commit 53d9dcf which adds an example model view application that just contains a combobox that lets the user choose between the available topics in a hardcoded bcf file. This application just serves as a proof of concept and guiding line over the next days.

commit 9005790 adds two functions that somewhat control the 3D view of FreeCAD. The first is vC.getIfcObjects() (vC stands for viewController) which compiles a dictionary of all objects in a document that have a IfcUID. Here a big thanks to Yorik who provided example code in his post. The second function is vC.selectComponents() which takes on a list of viewpoint.Component objects and adds every object with a matching IfcUID into the active selection.

July 4th: PI (Programmatic Interface) is nearing its finish, at least in the basic functionality. But onto the commits, and thus the work, I have done today:

commit 0d3d924 is rather small as it just adds a dependency check to ./bcfplugin/__init__.py. pytz is now also checked as dependency.

commit 66a73a8 introduces the function pI.addCurrentViewpoint() whose purpose it is to create a viewpoint object of the current view in FreeCAD. Currently only the camera position and orientation is read and stored in either a PerspectiveCamera or OrthogonalCamera object, depending on the type of the camera in the FreeCAD view. In the next steps also the highlighted components shall be detected and read in. But this depends on the ability of discovering the Ifc guid of a component in FreeCAD.

commit 36be8ce adds the option to add a complete new topic to the project. Alongside with a topic object, a new folder gets created inside the BCF file and a new markup.bcf file is created.

commit 539371f incorporates the modification of Topics and Comments in pI.modifyElement(). These two types have the speciality of containing both <ModifiedAuthor> and <ModifiedDate>. If a Topic or Comment object is updated then these two fields are automatically set/updated with it.

commit 75946db brings some testcases for pI.modifyElement().

July 3rd: Today I fully fixed the issue I found in writer.getEtElementFromFile() yesterday. The issue was rooted in the fact that there may be xml elements that occur in different parts of the hierarchy with the same name. For example <ModifiedAuthor> occurs once as child of <Topic> and once as child of <Comment>. In the algorithm for modifying elements first compiles a list of candidates, out of which the "to-update" element is picked by matching on either the children of the element, the text of the element or its attributes. Now the particular issue was that when someone already modified <Topic> and a <Comment> then <ModifiedAuthor> would have the exact same text. Due to insufficient selection of the candidates, both <ModifiedAuthor> elements (from Topic and Comment) made it into the list. That lead to indeterministic selection of the element to update.

This was fixed in commit 17c818e.

Then commit ebca39f added pI.modifyDocumentReference(), which, however, is made obsolete in part by commit 01fac66. In latter one I introduce a more general modification function pI.modifyElement(). It takes on an object of the data model, which is assumed to be modified. Next, the old element, referenced by original element, is deleted from file, the object in the data model is updated with the member variables and added again to the file.

commit 91ccac8 adds a backup and rollback system to all functions that alter the state of the open project.

July 2nd: Today quite a lot was done. commit 32213e3 updates README.md in feature/PI_retrieval to reflect the new plugin structure. commit 00d4758 adds pI.addDocumentReference(), which adds a new document reference to a given topic.

commit 55f6b2b adds pI.addLabel(), which adds a new label to a given topic.

commit e65fa52 is a rather interesting one: it introduces verbosity levels.

commit 0af3e03 adds the function pI.copyFileToProject(). Its purpose is to copy a file into the bcf file, so that it can be distributed alongside the bcf file itself.

commit b54acff adds full support for the camera settings in viewpoints. Now a camera setting of a orthographic camera as well as a perspective camera can be applied to FreeCADGui.ActiveDocument.ActiveView.

commit 9baa5fe made me aware of a bug in writer, which leads to improper modification of the xml nodes ModifiedDate and ModifiedAuthor. It also already sports the beginnings of the fix.

July 1st: As I already mentioned in the updated README.md on branch feature/PI_retrieval the source code structure had to change. This is what I have done in commit 4bcb152. Also in this commit I added the function pI.addComment(), which adds a new comment to a topic.

commit bf8df20 added a new function to pI.py. pI.addFile() adds a new file reference to the header node.

In addition to that I watched yorik's introduction video to BIM modeling.

June 28th: I gained a hell of a lot of understanding about the inner workings of FreeCAD. Even if I want to put my main effort right now on the pI (programmatic interface) the main goal for today was to find out how to set the camera of the active view to a specified position and orientation. Well I succeeded with these two main sources: forum-post by teobo and makro FCCamera by Mario52.

Apart from that I have rewritten the debug, more generally the output system of my plugin in commit 57c0b28. It now uses FreeCAD's Console to print outputs, if running inside FreeCAD. Otherwise the outputs will be printed to stdout/stderr. commit b401989 further moved the complete debug functionality into util.py, which makes kind of more sense than leaving it in project.py.

My new and gained knowledge of today, about setting the camera's position and orientation, got baked into a new function of the pI: pI.activateViewpoint()

The best comes at the end: I updated the wiki page on feature/PI_retrieval. It now explains how to use the plugin inside FreeCAD's python console. The associated commit is commit 2fdc65c.

At this point I want to give a great thanks to the efforts of qingfengxia and luzpaz. Their eBook on FreeCAD helped me a lot today in finding out how to modify the camera settings.

June 27th: Some development work happened today and some work with FreeCAD and IFC files.

Since today I pushed rather many commits with some sporting only minor changes, I will only mention the bigger ones below.

commit 25a0ee8 introduces the pI.getViewpoints() function. It takes a topic object and returns all viewpoints mentioned in the corresponding markup.bcf file.

commit fa5af15 modifies pI.getComments() in a way that it now also accepts an optional viewpoint object, in addition to the topic object, and returns a sorted list of comments mentioning that specific viewpoint.

commit e56747f adds the function pI.getRelevantIfcFiles(). It returns the list of files listed in the header node of markup.bcf.

Although commit 7192ca8 does not add that many new lines, it is quite significant. It adds the file BCFPlugin.FCMacro. The plugin is now already usable, to an extent, inside FreeCAD, without the gui, however. The wiki page will be updated tomorrow, still on branch feature/PI_retrieval.

To explain a bit more about the work I put into FreeCAD and IFC files: first I wanted to know how IFC files can be opened using FreeCAD, after that was accomplished, using IfcOpenShell, I searched for the IFC attributes (like the id of an ifc object) and where it is stored in the document. Result was: inside the class BuildingPart the member IfcAttributes exists which is filled with the attributes I want, at least I think so.

June 26th: Today a considerable amount of work was done in ./src/frontend/programmaticInterface.py. This file is beeing developed on the new branch feature/PI_retrieval. The first commit 1038b31 integrated the defaultValue member of SimpleElement and Attribute in every getEtElement() method. For more information please see the commit 1038b31 and its commit message.

commit 59d1ca8 changed the validation mode of xmlschema from 'strict' to 'lax', which means that a list of error messages, if there are some, is generated and returned with the decoded XML file, instead of throwing exceptions.

commit 2c88875 added to programmaticInterface.py (pI.py for short) the functions openProject() and getTopics(). Please see the commit 2c88875 and its message for more information

commit e31d3b3 adds getComments() to pI.py and prints out all validation errors if there were some. Also if some required node/attribute in a viewpoint.bcf file is missing then this file is skipped.

June 25th: Again some things happened outside of the git repo, like filling out the first evaluation form from summerofcode.withgoogle.com or reading more about the model view paradigm in Qt. Apart from this still a little dev work has taken place today: commit cf73654 renames Topic.refs -> Topic.docRefs and frontendInterface.deleteObject() now uses writer.addProjectUpdate() instead of writer.addUpdate(). Both accomplish the same, but former is safer to use.

commit 4d170a7 introduces many new comments of functions and already converts some comments to the official docstring format, as mentioned in yesterdays log entry.

commit 6f4b105 adds function writer.createBcfFile(). This function compresses the contents of the directory passed as parameter to a zip archive that complies with the requirements for BCF archives.

June 24th: I finally found the bug in the test suite for frontentInterface.deleteObject()! commit ab09e0a is the one in question for this change. In addition to that I started a little refactoring session and I am moving now to the official docstrings of python in order to generate nice documentation through pydoc. For the UI part I started to look into the model view paradigm, and how this is done in Qt.

June 22nd: I have written exclusively on the wiki page today as well as created a (hopefully) informative Readme that gives a crash course on how to use the plugin in its current state. Here you will find the wiki page now accompanying the plugin.

If you jump straigth to the repo you will find the new front page with the extended README.

June 21st: Today I added a few minor commits. The first beeing commit 3dcb227 in which I finally deleted the class Modification, which got split up into modification.ModificationAuthor and modification.ModificationDate. For more info please see the log entry of June 19th.

commit 8ec8c6f replaced the debug print instructions with the a call of the project.debug() function, in writer_tests.py.

After some debugging I added a tearDown() function to the test cases in writer_tests.py, done in commit c5cce73. Reason beeing that I noticed strange behavior after I added a test case for adding a whole new topic to a BCF file. Specifically most test cases failed if run together, but succeeded when run separately. Reason being that the extracted BCF file wasn't completely replaced for each new test case, instead it was merely updated. This meant that the added topic would be read in too by the following test case and thus invalidate some invariants.

commit f91b863 added a diagram of the basic structure of the plugin, for the purpose of the wiki page which I am currently writing.

In commit f7a4958 I added a whole new file frontentInterface.py with a new function frontentInterface.deleteObject(). Both currently reside on the branch feature_interface_deleteObject branch. This new function shall handle the complete deletion of an object, by that I mean the deletion from the file, through the writer module, and the deletion from the data model, through an also added function deleteObject() inside of Project.

Finally commit e561233 added a new test suite, intended for testing the functions of frontentInterface.py. It currently already sports 7 test cases.

June 20th: commit fed05f2 renames the Identifiable interface to XMLIdentifiable and adds a new Identifiable interface. The new interface is implemented by nearly all classes in the data model and assigns them, upon creation, a unique id. This enables an efficient search algorithm that uses an object's id to get the corresponding reference in the data structure. The Interface XMLIdentifiable is now only used to hold ids that are read in from the BCF file.

commit a0c4f8d implements the searchObject() function, in nearly all classes. This function empowers a depth first approach for finding an element. In addition to this function test cases were added to test it for proper function.

Locally I am currently working on the first part of the data model <-> frontent interface. I am implementing a deleteObject() function that deletes the object from the BCF file and from the data structure. But since it is currently under development I didn't already push it.

June 19th: Today most work got into thinking about how to do the interface between the data model and the GUI or the python interface for nonGUI mode respectively. Apart from thinking however I also pushed a major commit 8ceb3e8. It adds writer.modifyElement(), writer.processProjectUpdates() as well as helper functions. writer.modifyElement(), apart from writer.addElement() and writer.deleteElement(), also takes the old value of the modified element as parameter. This is necessary to find the correct element (attribute or simple element) in the xml file. writer.processProjectUpdates() has the purpose of iterating over a list of updates to the project object and calling the respective handler function (writer.handleAddElement(), writer.handleDeleteElement() and writer.handleModifyElement()). If some error occured during the update the errorenous update is returned, in case of success it returns None. Also in this commit I added a list writer.projectSnapshots which holds an arbitrary number of the latest n updates. This is supposed to fuel the undo operation, and will be used in the future. As always for a bit more of information please see the respective commit 8ceb3e8

June 18th: writer.deleteElement() is finished! (except for proper documentation) Finishing commit is commit 3765658. writer.deleteElement() now distinguishes between an identifiable element (one whose object is an instance of interfaces.Identifiable), a non identifiable element and an attribute. All have to be handled in a different manner to one another or can be handled in an easier way than another element type. Other changes that were implemented while writing on writer.deleteElement():

  • the interface interfaces.Identifiable now only holds IDs of type UUID, for more info please see commit a18599a.
  • project now implements a debug function project.debug() as it was morphed to the main place to handle debug prints. It also uses the inspect module to get the name of the calling function. For more information please refer to commit addc02e.
  • 6 test cases were added for writer.deleteElement(). You can find them in writer_tests.py
  • modification.Modification which formerly housed a member author and date is now split up into modification.ModificationAuthor and modification.ModificationDate. This makes it easier to handle in the writer-module. For more information see commit 59adbab.
  • util.py got two new functions: util.updateSchemas() and util.copySchemas(). These were added in commit 3765658 and are used to manage local copies of the schema files.

June 17th: As expected I had to put everything today into the project on university :/

June 16th: Today I paused. Tomorrow I will have to invest time into the project at university, so I don't know whether I come to do much work on FreeCAD tomorrow either. June 18th I will be back working on writer.deleteElement() again!

June 15th: writer.deleteElement() is not finished yet, work is still done locally. On master commit 9f04faf comprises some notable changes to ./src/bcf/writer.py. Most notably is the renaming and enhancing of writer.getContainingETElementForAttribute() as well as the addition of new testcases for this renamed function. But for more information please see the commit message as it is quite elaborate.

The current state of writer.deleteElement() is that elements, whose types inherit from interfaces.Identifiable, can be deleted.

Additionally to the first two points I thought about how to handle modifications of the data model. Should there be a separate function writer.modifyElement() or could it also be constructed out of writer.deleteElement() and writer.addElement()? Answer is: it could be constructed. But with the implications that either:

  • every change is written instantaneously to disk and the data model stays coherent with the bcf file. The disadvantag is that batchable updates are not possible and it may use quite a lot of CPU time.
  • Or for every modification a snapshot of the data model is stored, with the modified object in a list. This list is then processed chronological. This preserves the possibility of issuing batched updates, but might use a significant amount of memory.

June 14th: Today I finally finished the unit tests for writer.addElement(), for information on what it does please refer to the function documentation as it is quite extensive and the function can handle pretty much. The unit tests are to be found in writer_tests.py, and they were finished in commit c0e4317. Additionally to the unit tests this commit also comprises refactorization of writer.getContainingETElementForAttribute(), a bugfix in writer.getInsertionIndex(), a change in project.SimpleList and the addition of getEtElement() in project.SimpleElement. But for more information please refer to the commit c0e4317.

Then the second big commit is commit 647b684. In it I refactored the assignment of the Hierarchy.containingObject member variable of classes implementing Hierarchy. I moved it from the reader module to the individual constructors which makes more sense, I think.

Currently I am starting my work on writer.deleteElement().

June 13th: All work today was done on branch unit_tests. Today one commit, commit 24558c2, was added. In short: this commit adds two new test cases and rewrites writer.getInsertionIndex(). The result of this function is now the greatest index possible at which an element could be inserted. Now you will also find yesterdays work in the commits:

June 12th: Viewpoint objects can now be added, resulting in the generation of a new viewpoint file in the corresponding topic directory, for more information please refere to commit 2593bdb. commit 4de5078 adds the function writer.compileChanges(). It is not that long or complicated, but the most stuff goes on under the hood of the function call in line writer.compileChanges()#415. It results in a depth first search objects that don't are in the original state. Hence every data model class had to be edited. Currently I am working locally on unit tests for the writer.addElement() method for which I have 11 testcases planned. I will probably push them tomorrow upstream.

June 11th: with commit 645a0f0 I added support for all attributes that are optional, to be added (at least the ones defined in markup.xsd). Currently I am not supporting the addition of whole new projects, and viewpoints are not mutable, so once after they are saved they stay. Then locally I am currently implementing the addition of a whole viewpoint file. Probably tomorrow it will be merged into master and pushed upstream.

June 10th: writer.addElement() function is again further finished. It now has the capability of adding attributes to File elements in the header. For more information see commit da46aa4, also markup.HeaderFile was added, for more information please refere to the commit message. commit 78ac6ce introduces project.SimpleElement, project.SimpleList and project.Attribute. They are used to represent the values of simple elements, lists of simple elements or attribues respectively. But they also inherit XMLName, Hierarchy and State so they can be treated like any other representation of an element.

June 9th: In commit 2afab2d I implemented half of XMLName interface for all classes. This interface defines a property xmlName that each class inherits. By default this property is set to the name of the class, but the constructor of XMLName also offers the possibility to define a custom name (this is needed for the writer module). Second to the xmlName property it defines a member function XMLName.getEtElement(element) that shall receive an xml.etree.ElementTree.Element object and shall extend it with its properties, and return it again. It is expected that the returned element is schema conform and can be inserted as it is.

commit e013043 finally removed SchemaConstraint and its decendants, since it was actually unnecessary.

commit 3eeb7f8 added to the writer the functionality of adding objects of type comment into the corresponding markup.bcf file.

June 8th: I have worked on the writer.addElement() method. All work is still local and not finished. The overall strategy for the writer module is to read in the corresponding XML file using xml.etree.ElementTree, add the new elements in this structure and write it to the file again.

June 7th: Other than beginning a new blog post about the writer module and how I envision it, I didn't come to much today.

June 6th: Most work today was organisatorial: had correspondence with Matteo Cominetti as well as with Paul Deckers (a Product Specialist at the BIMcollab Support Team) about the topic of handling non conform schema files. This topic bugs me! For the writer module I am starting to write, I started a list that contains the elements that shall be updateable/addable and deleteable, will be available in a future commit. On to the development: commit d6c6cc5 I added an own class for the labels of a topic. This class (Labels) inherits from list and also inherits the Hierarchy interface, that I introduced yesterday. The initialisation of a Topic object is unchanged, in the init function the list of string labels is passed to the constructor of Labels. The inheritance from Hierarchy offers the reader the possibility of easily generating the path that leads down to the corresponding label element in the XML file. commit de38b48 adds the parameter guid to the constructor of Comment. Till today I overlooked it, which lead me to a pseudo problem. Without the Guid of a comment I would have had the problem of uniquely identifying the comment that shall be updated or deleted by the writer module. Additionally the commit de38b48 finishes the writer.getUniqueIdOfListElementInHierarchy() function that generates the hierarchy of a given element and checks if it contains an element that only occurs in a list, if that is the case then the unique id of that list element is returned.

June 5th: Today I finally followed the suggestion of @yorik and replaced my own code for getting the path to the temporary folder with the python module tempfiles, for more information see commit 5616fd9. commit ac589c8 is a small one, but with some repercussions. In it I added _viewpoint to the initialization sequence of the markup.__init__() function. Without it I wouldn't have been able to use the property markup.viewpoint. The amount the previous commit was smaller than usual commit c9f9ea4 is larger. In it I realized some conceptual stuff, like already implementing three interfaces in many classes, that will make the writer module easier to write and more maintainable. For this I added the ./src/interfaces folder. It is its own python package, and defines three modules/interfaces (atm):

  • State: represents the state of an object during the plugin lifetime
  • Hierarchy: allows an object to know the object it is part of and thereby somewhat implementing a doubly linked tree. For example consider an object of Markup. It probably has one or more objects of type Comment. Now each comment has a member containingObject, which in this case references the one object of type Markup that references it.
  • Identifiable: allows an object to define a unique Id, if it shall be uniquely identifiable. This is intended to be used mainly for objects that can occur multiple times. Like for example Viewpoint. Here the Id is the id defined in the XML file.

But please refer to the commit itself for more information, as it has a rather extensive commit message, describing the interfaces in more detail.

Please note: tomorrow I won't be able to do much. I try to throw in between one and three hours.

June 4th: In commit 0557bcc I fixed the issue where the Comment s didn't have a valid ViewpointReference object after creation. The member Comment.viewpoint should, after creation, hold a reference to a valid ViewpointReference object that was created in reader.buildMarkup(). Actually it was not implemented till today, I just left a TODO note for me. In commit 354d2c4 I added new testcases, all for testing the results of reader.buildViewpoint(). commit 0733b59 includes fixes of bugs I became aware of during testing reader.buildViewpoint(), as well as the implementation of __str__() for some more classes, which helped during debugging. Then commit d6cb41c also contains bug fixes and debug prints were remove, but see more in the commit message. And finally I added the file ./src/bcf/writer.py with some dictionaries that I will need for the writer module.

Last but not least one thing I learned today using git: don't rebase onto master. Rebase master onto some branches but not the other way around. This makes things messy.

June 3rd: A new blog post is ready, this one elaborates a bit on the ideas I have in regards to handling non schema conform BCF files. The branch unit_tests got a few new commits, mainly adding test cases for the reader.buildTopic() and reader.buildComment() functions, for more info see commit c94d812. commit 7a31462 contains bug fixes for bugs that I became aware of due to the newly created test cases. Offline I thought for an extended period of time about how to structure an update in place approach to writing the bcf file, rather than writing the whole file at once. For this a new blog post is coming this week.

June 2nd: I have written on a blog post about how to handle non XSD conform BCF files (which is not finished, yet). Otherwise I paused development. Tomorrow it is gonna be picked up again!

June 1st: Today not much work was done, but here is a little summary. I implemented the __eq__() function in every class I defined, see commit 0305754. This shall help me in the future when I want to write unit tests. In reader.py I inserted a rather long comment about how the buildX functions, like what they do, what they expect and what they return. Since all behave the same I have written one big comment to document all of them, but see commit a05e22b. The branch feature_read_viewpoint got merged into master, for testing I will create a own testing branch. And I created a package out of the src/bcf directory that just exports reader.py and writer.py (latter one I still have to write), for details see commit 015c2f6.

May 31st: Today all work was done on the feature_read_viewpoint-branch. Most important the function reader.buildViewpoint() is finished and with it the last step was completed to being able to read in a complete BCF file, given it is validated successful against the XML schemas. For more info see commit 2922d71. To test the reader "module" I created two new topics in src/bcf/test_data. One is complete in the way that it defines at least one element for every node in all files specified by the corresponding XSD file. The third topic just has a complete header element in markup.bcf. For more info see commit 7fa127a. Now ViewpointReference has a reference to the corresponding Viewpoint object. The inheritance approach thrown away because after reading in all ViewpointReferences from markup.bcf the Viewpoint objects would have been created, but with no relation to their super class. This meant that an object of the former could not have been used as an object of the latter without recreation of this object. That further would have complicated the code, and made it hard to understand and maintain. Therefore composition was chosen in favor of inheritance. For more info also see commit 2922d71.

May 30th: Added the folder src/bcf/test_data that is intended to contain test data for testing during development, but not primarily for unit tests now. Currently it contains an example compliant with the schema files. For more info see commit 6fb72f5. In commit 30b998d I changed the type of every variable associated with SchemaConstraints to an elementary type. Reason being that the extensions.xsd file (as my understanding goes) is intended to be specified in the xml file itself as ... well, an extension to the existing schema. Finished function reader.buildMarkup(), although still a "#TODO" comment is above the function header. Also in commit aa04598 the class DocumentReference and BimSnippet was added. Last but not least: started working on reading in viewpoint.bcfv. But not finished, that is why this development is still on the branch feature_read_viewpoint. For more info refer to commit 154630d.

May 29th: Already started refactoring a bit in reader.py. Went away from using ZipFile objects when operating on the zipFile to the extracted version of the zip file instead. Written function reader.readInFile() that shall read the complete BCF file into the data structure (see commit 9a79162). During testing I noticed that the example BCF file from the bimcollab website is not valid becaus it defines an empty node Header in markup.bcf of topic ebb1a8bf-6d1d-4aad-a875-61ad3cc40d30 which is prohibited by markup.xsd of BCF version 2.1.

May 28th: Created a new blog post about the way the schema constraints are handled now (schema constraints revisited). Change configuration file of the blog to reflect the current project, instead of the FreeCAD development blog. Started a unit-test suite, see branch unit_tests ./src/tests, and test cases written for reader.buildProject(). Written function in reader.py that parses bcf.version and returns the version number as string, see commit cccde6a

May 27th: Complete the python representation of the class diagram commit 0a1081b. Also advanced to the point where a .bcf can be opened, every XML file be validated (in theory, only tested it with project.bcfp yet) and the contents of project.bcfp are can be written to an object of Project. For more details refer to following commits:

May 26th: Update Comment in class diagram. For more info see commit 1c34ad9, and create mockup of the plugin interface. Write part one of the class model in python see commit 31ef931.

BCF-Plugin GSoC 2019

This blog post shall give detailed information about the project GSoC 2019 BCF-Plugin. The following sections will elaborate on a few key aspects of the project including:

  • the initial vision and with it the reason why it was done
  • some information about the BCF standard
  • what exactly has been accomplished in this project
  • main hurdles that cost a considerable amout of time
  • ideas for improvment.

Vision

BCF stands for BIM (Building Information Modelling) Collaboration Format. It is used in the industry by architects to collaborate effectively on one building model. Collaboration in BCF terms means that, apart from the building model a separate BCF file is handed over to the other architect(s). Using a tool that supports the standard, these parties can then add issues/topics, supply these with context in from of comments, snapshots, addtional documents and more.

The intent behind the FreeCAD BCF-Plugin was to also integrate support for BCF into FreeCAD, and thus adding FreeCAD to the list of BCF supporting CAD tools.

BCF-Standard

During the course of the project I have written quite a lot of documentation. This includes already a primer on the BCF standard that is more expansive than the one above. So for more information on BCF please refer to the project's wiki page.

Outcome

FreeCAD can be used in two modes:

  1. GUI-mode: in this mode FreeCAD can be used as Qt application.
  2. nonGUI-mode: is composed of a python console that gives access to all aspects of the opened file.

Apart from these two modes, a plugin framework was implemented which is able to run standalone python applications within the FreeCAD context. Through the suggestions of my mentors (@yorik and @hardeeprai) I developed the BCF integration as python application that accesses FreeCAD's state via the its python interface. The plugin, like its host application, can be controlled either via a graphical user interface or via the python console, proviced also by FreeCAD.


Complete integration of the BCF standard in the timeframe of just about three months would have required more than one person, presumably. Thus, here the aspects that are accessible through the GUI are listed (practically everything can be done in nonGUI-mode, since complete access to the parsed BCF file is given to the user):

  • BCF files are read into an internal data model
  • a topic can be chosen from the list of specified ones.
  • most metrics of a topic can be edited in a separate window.
  • the list of additional document references of a topic can be displayed
  • relations between topics can be explored
  • comments can be edited/added/removed (this is the part which was most emphasized during development)
  • the snapshots of a topic are shown,
  • as well as a list of all available viewpoints.
  • a snapshot can be opened in full resolution
  • a viewpoint can be applied to the object viewer of FreeCAD.
  • the current state of the bcf file can also be saved to file again.

Hurdles along the way

In this section I want to list some of the most significant hurdles I experienced during the last three months.

The biggest of all probably was supporting BCF files that do not comply with the standard. Unfortunately it is intended by the BIM collaboration that vendors of BCF tools can add their flavor to the BCF files created by their tools. This posed quite the obstacle in two ways:

  1. What to do with additional and malformed information, that is not defined by the standard?
  2. How can BCF files be opened and written again without losing potential information that did not comply to the standard?

The approach taken for (1) was to just omit data that did not comply to the standard during reading it in. However, this had significant consequences for (2) as it ruled out the easiest strategy for writing a BCF file: just serializing the whole data model from memory to disk. Thus, (2) was solved by using an update write strategy. Thereby the BCF file is opened in a temporary directory, and every modification, done by the user will be commited to it as an update. This update approach was designed to also support batch processing (i.e.: write multiple updates at once instead of immediatly when they arise).


The second major hurdle was the comment list in Qt. My background in programming does not include user interface programming and thus I had no experience with Qt. Items in the comment list show the three major aspects of a comment at once:

  • the comment text itself
  • the E-Mail address of the person that either created or, if the comment already was modified, modified it at last, and
  • the date of creation or last modification respectively.

These three aspects are split up into two rows, separated by a thin line, where the comment fills the first row and comment and date share the second row. The difficulty here was to properly implement custom list elements in the Model/View Paradigm of Qt. The current implementation uses, apart from a custom model and view, a delegate. It is responsible for drawing the list elements as described above and for handling modifications to the list elements.


The last, in comparison, rather small hurdle was the correct handling of temporary directories. In theory it is quite easy:

  1. a TemporaryDirectory object is created like in the code example below. This object is itself a context manager and automatically deletes the temporary directory again when it is garbage collected.
import tempfile
tmpDir = tempfile.TemporaryDirectory()
  1. when you are finished run del tmpDir and the whole directory will be deleted again.
  2. during runtime of the application store the object in the application global space.

That is the theory. However in combination with Qt's event handling mechanism the assumption that the application global variable will have the same state when reading it in an event handler and in normal operation does not hold anymore.

In my case I had a global function getSystemTmp() that creates a temporary directory only on first invocation, and on each subsequent call return the already created directorie's path. However, it twice created a new temporary directory. The second one was created when asking the user for his/her E-Mail address before a change is commited.

In the end this context problem was solved by a separate file in the system's temporary directory. It holds only the path to the created temporary directory, which serves as working directory. Thus, now getSystemTmp() creates the file on the first invocation and on each subsequent call reads the one line from the file and returns it.

Outlook

Some things are still missing at this point in time (2019-08-19) Following a list of things that the plugin still needs to serve the user best.

  • Apart from opening and saving an already existing BCF file, the creation of a completely new one is probably integral to a BCF tool.
  • Adding new topics to a project
  • Link comments to viewpoints
  • Add new viewpoints
  • Create snapshots
  • A list showing referenced documents with their respective paths is already implemented, but it would be nice if they also could be opened right from the plugin with a double left click or something like that.
  • Create relations between topics in form of <RelatedTopic> nodes in the markup.bcf file.

At this point I want to thank my mentors Yorik van Havre and Hardeeprai (unfortunatel I don't know if his real name is written this way, because I only knew him by his user name on the freecad forum), for giving me the possibility to contribute to FreeCAD in a meaningful manner!

Writing non Schema Conform BCF Files

Wait would you write files with the intention that they don't conform to the provided XSD files? ... you might ask by just reading the title. No I wouldn't. As I introduced in my last blog post (Handling Non-Conform BCF Files), that was not an update of the dev-logs post, I introduced the need of supporting files from vendors that might do some things differently. I ended this last post with a pros and cons list about two approaches that I thought of. In the meantime I decided for latter one.

Recap

The second approach, let's call it update approach, extracts the BCF file to a temporary folder, reads the contents into memory (that are conform to the corresponding XSD file) and every time the state in memory is changed the updates get written to the dicrectory structure in the temporary folder. At the end, when the plugin is closed and its lifetime ends, this directory is compressed again and stored in its original location. The update approach thereby does not alter any data that the plugin didn't load into memory, it works "around" it.

I think this is the better appraoch, since no exceptions to the rule have to be made for every speciality that a vendor might incorporate in its files. However, the writer module has to have an understanding of XML baked in, in order for it to be able to update these files in place.

In the next section I want to elaborate a bit more on how I envision this writer module.

Writing actual stuff

At the beginning of every writing operation it has to be known whether an existing element shall be modified, a new element shall be added or an existing one shall be deleted. Following you will find my thoughts, uncompiled, as I have written them down onto paper. First will be the handling of additions.


How can additions be handled?

Every element class (a class that directly represents a XML-Element like Comment) shall have a function that returns the XML-Text. This text is then going to be inserted at the right place by the writer module. The writer module has to crawl through the complete data model to check/compile a list of objects that were added, changed or deleted. Then for each added object it calls the XML-Text function to get the corresponding XML-text for each element. A second function shall give the following information about the position at which it should be inserted (could also be done with a dictionary, that way the data model stays clean):

  • the containing element: the containing element of Comment is Markup
  • the relatie order: a list of elements that preceed the current one in reverse order. So the immediate predecessor is first in the list. The last element of the list is the containing element.
  • if the new element shall be added inside an element that might occur more than once (e.g.: ModifiedDate in Comment) then also a unique id of that list element has to be retrieved.

The writer then searches for the containing element in the file and then goes through the contents of the containing element, searching for the one with the least index in the relative order list. The text is inserted right after the closing tag of the chosen element.

Any references should not be handled/checked in the writer module (e.g.: that the viewpoint guid in the to be written comment object belongs to an actual existing viewpoint). Such consistency checks are assumed to be handled in the GUI or the programmatic interface respectively.


How can updates be handled?

An element either has to occur only once in the corresponding file, or be identified uniquely by some attribute. Without one of these two constraints the identification of the right element in the XML-file becomes extremely hard or cannot be done deterministically. An object shall be flagged MODIFIED iff (=if and only if) a non-list child element was changed. A change of a list element translates either to an addition or deletion, or a change of an element that can be identified deterministically, as described above. In which case the object, containing the list with one or more items changed, shall not be flagged MODIFIED. Each object shall then contain a list of changed objects, oblivious of whether it translates to an attribute or element. for each of these objects the writer module has to gather following information:

  • element hierarchy: ModifiedDate in Comment would have ModifiedDate -> Comment -> Markup as hierarchy list.
  • unique id of every list element that is in the hierarchy (like with additions)
  • whether it translates to an attribute or an element

The writing module shall collect a list of all elements that are flagged MODIFIED. For each element the writer module shall collect the amount information listed above. It then follows the element hierarchy of the object in reverse order till it reaches the changed object itself. At this point the distinction between attribute and element has to be made. For both cases the following two steps happen:

  1. the respective substring of the element has to be found and
  2. be replaced entirely by the XML-text generated from the object

How can deletions be handled?

The writer module compiles a list of all elements to delete. For each element the following information shall be gathered:

  • which file is the element contained in
  • unique id of the element (if available)
  • name of the element

The writer module then searches for the unique id in the specified file that also matches the element name and deletes it from the start to the end tag.


How shall the interface to the writer look?

The above mentioned crawler has the advantage of alos supporting writing the file only once. Therefore also being somewhat resource saving. The writer shall have a function writeChanges(). This function receives an object of type project which denotes the starting point of the search. Every object shall have a variable of enum type. This variable is allowed to take on one of the following four values:

  • ORIGINAL
  • MODIFIED
  • ADDED
  • DELETED

For each of these states the writer maintains a separate list. Each state has to be handled differenty during writing. It shall also be possible for the user to just issue an update of all of these four lists and then subsequently manually decides the point in time when updates shall be written. => updatechangdLists() -> writeChanges(). Since writeChanges() now has to fulfill two functions: once triggering updateChangedLists() and the other time just take these lists as they are, its argument shall be optional. For an addition, deletion, update it is important to know in which folder the to-update file is stored. For this a get topic function shall be implemented.

Handling Non-Conform BCF Files

This post shall just collect some ideas I had about handling BCF files whose contents may not be (in part) conform to the XSDs supplied by buildingSmart on github.

Why is there a need for that?

Well, during testing with one of the two BCF files I currently have (here the link to the first one to download) I wanted to validate each file I read in, before I read it in, to be (mostly) sure that my code does not throw unexpected exceptions. But already in markup.bcf of the first topic the library I use, xmlschema, threw an XMLSchemaValidatorError. The reason was that the node <Header /> was defined, but it was defined empty, which is prohibited by markup.xsd. So as quick fix I deleted the <Header /> node altogether to be able to test my code somewhat. But now on to the next section.

How does xmlschema report parsing errors?

Before we can go into the problem solution, a little investigation of the library I am using has to be done. We need to know how it handles the nodes and attributes that could not be parsed/decoded against the corresponding XSD file. For parsing/decoding xmlschema defines the following three validation modes:

strict: Schemas are validated against the meta-schema. The processor stops when an error is found in a schema or during the validation/decode of XML data.

lax: Schemas are validated against the meta-schema. The processor collects the errors and continues, eventually replacing missing parts with wildcards. Undecodable XML data are replaced with None.

skip: Schemas are not validated against the meta-schema. The processor doesn't collect any error. Undecodable XML data are replaced with the original text.

For the ones, like me, that got confused by the first sentence of the first two modes: every schema that is read in is itself validated against a schema, the meta-schema, before it can be used. These modes can be used separately for reading the schema file and for parsing/decoding a XML file. So for example it is possible to read the markup.xsd in strict mode, thereby validating it against a meta-schema, but then parsing/decoding the markup.bcf file in lax or skip mode.

In what way can errors be handled?

Given the modes in which XML files can be read in by xmlschema I could only think about the following two possibilities on how to handle possible errors:

  1. Just ignore the node that could not be parsed/decoded against the corresponding schema file. Easy to accomplish with lax mode.
  2. Read in the plain XML text for that node, store it somewhere in the data structure and when writing it all out again just insert the plain XML text into the right place. Only possible in skip mode.

The main drawback of number 1 is clearly that the data, that couldn't be read in, would be lost, or only exist in an earlier version of the BCF file, after the datastructure got written. It would certainly be easy to implement but does not offer the user great value if his or her data get lost. This is, however, on the premise of reading in the file once and writing it once after all changes have been made in memory.

The second option offers a possibility to circumvent the lost data problem. The not-parsable elements just get read in plain.

But there may be another option that could use lax mode without the data loss. This option is another approach to writing the contents of the datastructure to the file. Originally I planned to do it in one sweep when the user presses the enter key or the save button for that matter. But then hardeeprai from the freecad forum suggested, to another topic, that I write changes right after the were made. So effectively implementing an update driven approach to making changes permanent.

So now there are two possible approaches to handling errors that each use a different validation mode. In the next section I want to list some thoughts I came up with for each one of these approaches.

Approaches: Pros & Cons

Reading once and writing once

As already established, skip mode must be used here with xmlschema. For every member in the data structure it has to be noted somewhere if it could be read in or not. That would indicate whether the node holds the acutal value of the intended datatype or if it holds the plain XML text as string.

Advantage: writing the datastructure is fairly easy

Disadvantage #1: every class, that represents part of the XML file, has to be changed in order to house the additional information about every member or a whole new datastructure has to be created for that.

Disadvantage #2: the type annotations become invalid, if this approach is used, which leads to confusion by the ones reading the code.

Disadvantage #3: the module that reads in the XML file into the data model becomes much more complex, thereby not aiding the understandability of the code.

Updating the file on the fly

Here lax mode can be used with xmlschema. Still the whole file has to be read in once, but the parts that could not be parsed/decoded stay in the file and are not read in. To really be able to update the original file, the datamodel has to accommodate information for every element if it is new or updated, and therefore shall be written or not. The overhead here is comparable to the first approach but every class just needs one additional member to signify a change or that this object was newly created.

Advantage #1: (as already mentioned) not so much overhead in the data model compared to the first approach.

Advantage #2: The user is not exposed to the danger of losing unsaved changes, since it is updated every time a new change is committed to the data representation.

Disadvantage #1: the module that writes the data to the BCF file has to have light parsing capabilities or a XML awareness respectively, in order to be able to edit XML files directly. This does not have to be the case with the first approach, since we could utilize the xmlschema library.

Disadvantage #2: Objects that get deleted must not be deleted right away from the data model. Rather a flag has to be set that it is "deleted" and the corresponding part in the file has to be removed as well. After that it can be removed from the data model.


I am gravitating more to the latter option, since it does not interfere with the type annotations, and therefore does not complicate the whole type situation in the data model. What I still have to face, however, is the XML aware writer module. This is going to be fun!

Schema Constraints Revisited

In an older post, one or two weeks back, I've written about a design issue I was facing when trying to integrate the constraints of extensions.xsd into my class model. extension.xsd defines possible values for the XML nodes:

  • TopicType
  • TopicLabel
  • TopicStatus
  • SnippetType
  • Priority
  • Stage

I wanted to ensure that the variables, representing let's say TopicType, can only assume valid values, as defined in extension.xsd. My first idea was to just define Enums following the schema. Generally this idea is fine but it doesn't do so well with version changes. A new version of the BCF-XML standard would then possibly require more maintenance work than necessary. My aim with this BCF plugin for FreeCAD is it to create a data model that requires as little maintenance work as possible, although the code still has to be adapted to support any new version.

From the maintenance point of view hardcoded enumerations were not really an option.

The next idea was to create something like dynamic enumerations. These would have the advantage that the constraint is enforced (i.e.: a variable holding an enumeration value can only hold a valid value as defined in the enumeration itself). I knew it was possible to change a class object (not the object of a class) during runtime and add or remove members from it. I thought maybe this is also possible with classes that derive from Enum. In python to create an enumeration you just have to define a class like this:

from enum import Enum

class Example(Enum):
  VAL1 = 1
  VAL2 = 2
  ...

The first obstacle was that classes can only derive from a class (that itself derives from Enum) iff they don't introduce new members. Otherwise this would break some invariants the python interpreter has over enumerations. The second obstacle was a logical implication of the first: if subclasses couldn't add new members to an enum-super-class then it is also not possible to add new members to enumerations during runtime.

So another solution had to be thought of. I maintained the basic structure I had envisioned for the dynamic enumeration solution. Namely one superclass called SchemaConstraint and one subclass for each of the above mentioned XML nodes. Every piece of code that wants to represent one of these XML nodes would store an object of one of the subclasses in a variable of type SchemaConstraint. The actual value of the object is stored in a class variable, value, that holds just any value of type string. This approach of course has the downside that basically any value can be assigned to the variable value, if not restricted. In order to change this I made the class variable value a property where the setter checks whether the new value is actually a valid value.

Now the question arises: How does the setter function know which value is valid? Every node has different valid values!

To solve this issue I introduced, for one, the list validValues into SchemaConstraints (value does also reside there) and, for the other, a static function called parseConstraints(elementName). This static function reads the valid values for the XML node elementName from extension.xsd and returns them as list. parseConstraints() is intended to be called from the __init__() function of each one of the sub classes (e.g.: TopicType.__init__()). To make things clearer I put a small code example below showing the classes SchemaConstraints and TopicType as well as all the members of both classes.

class SchemaConstraint:
  def __init__(self, value, validValues):
    self.value = value
    self.validValues = validValues

  @property
  def value(self):
    ...

  @value.setter
  def value(self, newValue):
    ...

  @property
  def validValues(self):
    ...

  @validValues.setter
  def validValues(self, newValue):
    pass

  @staticmethod
  def parseConstraints(elementName):
    ...

class TopicType(SchemaConstraint):
  def __init__(self):
    validValues = SchemaConstraint.parseConstraints("TopicType")
    super(TopicType, self).__init__(validValues[0], validValues)

One important thing to note here is that the setter method of validValues has just one line which contains pass. This is done in order to ensure that the set of valid values does not get overwritten, at least not that easily.

This solution has, for one, the advantage of being adaptable, in case new values get added to the list of valid values and, for the other, through the setter function of value it can be enforced that the state of an object is always valid.

Learning XSD Syntax

The main takeaway from the XML Schema Definition Tutorial I went through today was the following:

So the node <xs:sequence>...</xs:sequence> is part of the definition of a complex type, it is also called an indicator. What it does is that it tells the XML checker that the elements specified inside have to occur in the same order in the XML document to be valid. Also it specifies what elements may be contained in the complex node.

This greatly improved my understanding of visinfo.xsd which describes how a viewpoint.bcfv file is built. As I learned during design, the representation of a viewpoint is almost as complex as the representation of the remaining contents of a BCF file.

I thought about how I coudl read the contents of a BCF into this file structure. As my brain always does, it thinks through the whole process, from checking if the given file is valid till the point where all the data resides in an object in memory. This makes me prone to "reinventing the wheel" as I also design a XSD parser/checker in my head which I then would use to check the given files before reading them in. But a quick google search brought up a better option, namely an offical python library for checking XML files against a schema.

Last but not least: I am pleased to announce that the first draft of the class diagram is ready. You can find it here if you are interested.

Advancing to the Viewpoint.bcf part

In the forum post yesterday, Yorik pointed out that, although part of the specification, the BimSnippet does not have to be treated with high priority. He also referred me to Bernd and Duncan in case of questions arising. As advised, I will consider the BimSnippet in the data model but probably won't implement it right away.

The progress I made today was mostly in understanding the BCF specification in regards to the viewpoint.bcfv file. Multiple viewpoint files can be contained within the folder of one topic. At the beginning of the day I thought that the data model would be finished rather quick but as it turned out the documentation on it is not as concise as one might wish it to be. To be more precise: the whole section about the "Components" element is a bit confusing. It lists multiple "properties", like Selection and Visibility. My intuition says that Selection contains a list of components that are selected, and Visibility specifies whether these components shall be visible or not. But in reality each contains a list of components onto which the values of the attributes of the respective property shall be applied.

This is a bit confusing at first, and still is for me. Because of this ambiguity I decided, for the further data model design, that refer to the related XML schemas rather than the written documentation. For example here you can find the schema for the viewpoint.bcf file.

But since the last time I have worked with XML schemas was in vocational school (I think 2012) I have to refresh my memory on it. For that I will go through the relevant parts of the W3CSchools: XML Schema Tutorial. Tomorrow.

Representing markup.bcf in UML

Today I mostly finished the UML class diagram in regard to representing the data of the markup.bcf file. One think I found kind of interesing while creating the diagram is that in the BCF-XML-Documentation There is no link between a topic and a comment, so no id of one is stored in the other. That didn't make sense at first because I thought that one markup could have multiple topics, which is not the case.

However one think what is not that obvious to me is the case with the BIMSnippet. On the documentation page it says that: "BimSnippet is an additional file containing information related to one or multiple topics.". This would imply that there would be more topics inside one markup.bcf file, because BimSnippet is, as I understand, also just a node inside markup.bcf.

Regarding the open question from the previous post: How to represent constraints defined in extensions.xsd; I thought about it a little today and came to the conclusion that it would be the best approach to go with the last option I listed. That means that for every node, for which constraints are listed in extensions.xsd, an empty enumeration/class is created which then, during runtime, gets populated with the valid values.

UPDATE (2019-05-13): The inconsistency is resolved. The BimSnippet node just references one file that does not have to be inside the topic folder.

How to represent extension.xsd constraints

So by now this repository exists, which already houses two example BCF files in examples/, supplied by yorik. This repository is intended to be the main development repository. I started a little UML class diagram, which you can find in the folder doc/ in the root directory of the repository.

Note however, the class "Template" is just used for copy and paste. I used it to set the font to Sourcecode Pro, which was kind of tedious, and since I didn't know if these font changes translate to newly created classes too I have gone with the copy&paste approach. The classes presented in this diagram are all for representing the data from the markup.bcf file (except for the class "Project"). For the creation of these class diagrams I use Dia.

The current problem I am facing is: how to represent the constraints given in the extension.xsd file (a link to it here). The most constraints listed in this file are just listings of valid values for some nodes or attributes. What I want to do is to force the user of this classes to use one of the valid values and not have to implement extra code that checks each time the value is set or written to a file for that matter.

So what possibilities do I have to choose from?

Probably most intuitively would be to create one enumeration class (python reference) for each attibute or node listed in extensions.xsd. The main disadvantage her ist that these enumerations have to be maintained by hand, in the case of a change to extensions.xsd. That makes this first intuitive option quite unattractive.

The second option that comes to mind would be to parse extensions.xsd and then programmatically define the enumerations. This is possible due to the dynamically typed nature of python. However this also implies that the classes, containing a representation of an affected attribute or node, also have to be modified every time the plugin starts and extensions.xsd is parsed. This is because the classes, of which type the class variables are, do not exist before runtime.

A third approach would also be possible where one enumeration class is written, before runtime, with no values. Then during startup extensions.xsd gets parsed and the valid values are added to the enumerations. That however has one drawback: if the standard gets updated, and a new constraint is put upon another field then an according class would have to be created manually.

In the end I still have to think about how it is done best, and what is the most maintainable version. Already I tend to latter option, because when the standard changes, most probably also the implementation of this plugin has to change.

BCF Reference Collection - GSoC 2019

Following is a list of all references that were mentioned in the forum. I split them into two categories (for now): Documentation, Implementation Reference. In former category standards and definitions are to be found (e.g.: BCF-XML Documentation). The category "Implementation Reference" shall hold links to examples, tutorials etc.

This list of references is most likely to grow over time.

Diskussion

  1. Forum topic on BCF

Documentation

  1. How to Get Most out of OpenBim
  2. BCF-XML Documentation
  3. FEM Python coding standard: coding guidlines that I want to adhere to during development (written by PrzemoF).

Implementation Reference

  1. BIMBots-FreeCAD: standalone plugin created by Yorik, serves as example on plugin programming in FreeCAD
  2. QTDesigner Tutorial
  3. Matteo Cominetti: Author of the GNU BCFier plugin. May be a useful contact on questions regarding BCF definition.
  4. BCF Server and Forum: "[...] Wordpress plugin to create a 'BCF topic server' from your wordpress installation [...]"