From 94c228973d2f4cf8584a6558ba239ba0ff11e5d9 Mon Sep 17 00:00:00 2001 From: "Zheng, Lei" Date: Fri, 5 Jul 2019 08:39:54 +0800 Subject: [PATCH] App: API changes for document recompute/save/restore/import/export This patch adds support of recomputation with external linked object, as well as external document auto loading and partial loading. Application: * Modified new/openDocument()/signalNewDocument to choose whether to signal GUI for creating a view for the document. This makes it possible to suppress view creation when opening external documents. * New API openDocuments() which does the actual job of loading the document together with any external dependencies. There are afew extra arguments to allow setting FileName property differently from the actual file path, which are required when auto loading dependencies during document recovery (with future patch to Gui::DocumentRecovery) * openDocumentPrivate() is an internal helper for opening individual document. * New signalStart/FinishOpenDocument to be signaled before and after opening a document. There may be multiple depending documents actually opened in between these two signals. * New signalBeforeRecomputeDocument signaled before recompute a document. * New API addPendingDocument() for use by external capable link properties' to queue up external documents. * isRestoring/isClosingAll(), for convenience status reporting. Document: * signalFinishImport/RestoreObjects, new signal triggered after imported or restored all input objects * signalBeforeRecompute, signaled before start recomputing this document * Modified signalRecomputed with additional recomputed objects, this is to make it more efficient for Gui::TreeWidget to check recomputation result. * signalSkipRecompute, signal to inform which objects are skipped during recomputation because of their owner document SkipRecompute setting. * restore/save/read/writeObjects() modified to suport partial loading. See [here](https://git.io/fj6PY) for more information. * afterRestore(), internal function called to finish restore. The function is separated from restore() because there is quite a few critical steps needed to fully restore a document with external linking. See [here](https://git.io/fj6P4) for more information. * DocumentP::_RecomputeLog is modified to store more accurate object recomputation error, including those happened during restore/import. * isExporting(), new API for checking if an object is exporting. External linking properties will use this function to decide how to export. * copyObject(), modified to support external linking objects, and accepts multiple input objects. * moveObject(), modified to support arbitary object moves. The original implementation may cause crash if undo/redo is enabled. Furthermore, because the original information fakes the object's deletion to break its dependency, it does not work for objects that may auto delete their children when being deleted. The new implementation copy the object, and than paste it to the other document. It then deletes the input objects from the original document. In case of recursive move, it only deletes the depending object if it has an empty in list. * importLinks(), new API to import any external object linked by the input objects into this document. It will auto correct all link references after importing. * getDependencyList/_rebuildDependencyList(), these two APIs are unified and implemented by an internal function _buildDependencyList() with a new algorithm to handle external objects. The returned dependency list will now include objects from external documents. In case of cyclic dependencies, getDpendencyList() will report the actual objects involved in dependency loops. * mustExecute(), new API to check if there are any object requires recomputation. This function will call _buildDependencyList() and check for external objects as well. * addRecomputeObject(), new API for marking changes during document restore. It only marks the object but does not actually recompute them for performance reason. One use case is for geo feature to request for recomputation to generate geometry topological names. * recompute(), support partial, external, and inverse dependency recomputation. Improve error handling during recomputation. See [here](https://git.io/fj6PO) for more information. * _recomputeFeature(), suppoert user abort. * getDependentDocuments/getInList(), new API to obtain an optional dependency sorted list of depending documents. DocumentObject: * Add various ObjectStatus flags * isExporting/getExportName(), return a safe name for exporting, in the form of @, which is guarrenteed to be unique. Various link property will save linked object using this name if the the linked object is exported together with the owner object, see [PropertyLinkBase::restoreLabelReference()](https://git.io/fj6XO) for more information. * recomputeFeature(), add option to recompute this object together with all its dependent objects. * canLoadPartial(), new API for [partial document loading](https://git.io/fj6PY). MergeDocuments: * Move object name mapping logic to various link properties. See Base::Sequencer: * Add new API checkAbort() for checking user abort. --- src/App/Application.cpp | 257 +++++- src/App/Application.h | 58 +- src/App/ApplicationPy.cpp | 86 +- src/App/Document.cpp | 1427 ++++++++++++++++++++++--------- src/App/Document.h | 109 ++- src/App/DocumentObject.cpp | 39 +- src/App/DocumentObject.h | 28 +- src/App/DocumentObjectPy.xml | 20 +- src/App/DocumentObjectPyImp.cpp | 32 +- src/App/DocumentPy.xml | 93 +- src/App/DocumentPyImp.cpp | 173 +++- src/App/MergeDocuments.cpp | 9 + src/App/MergeDocuments.h | 2 + src/App/Property.h | 29 +- src/Base/Sequencer.h | 3 + src/Gui/MergeDocuments.cpp | 4 + 16 files changed, 1866 insertions(+), 503 deletions(-) diff --git a/src/App/Application.cpp b/src/App/Application.cpp index 846db470ef..fb1c12c4b1 100644 --- a/src/App/Application.cpp +++ b/src/App/Application.cpp @@ -221,8 +221,9 @@ init_freecad_module(void) #endif Application::Application(std::map &mConfig) - : _mConfig(mConfig), _pActiveDoc(0), _objCount(-1) - , _activeTransactionID(0), _activeTransactionGuard(0), _activeTransactionTmpName(false) + : _mConfig(mConfig), _pActiveDoc(0), _isRestoring(false),_allowPartial(false) + , _isClosingAll(false), _objCount(-1), _activeTransactionID(0) + , _activeTransactionGuard(0), _activeTransactionTmpName(false) { //_hApp = new ApplicationOCC; mpcPramManager["System parameter"] = _pcSysParamMngr; @@ -355,6 +356,11 @@ Application::~Application() /// get called by the document when the name is changing void Application::renameDocument(const char *OldName, const char *NewName) { +#if 1 + (void)OldName; + (void)NewName; + throw Base::RuntimeError("Renaming document internal name is no longer allowed!"); +#else std::map::iterator pos; pos = DocMap.find(OldName); @@ -368,9 +374,10 @@ void Application::renameDocument(const char *OldName, const char *NewName) else { throw Base::RuntimeError("Application::renameDocument(): no document with this name to rename!"); } +#endif } -Document* Application::newDocument(const char * Name, const char * UserName) +Document* Application::newDocument(const char * Name, const char * UserName, bool createView) { // get a valid name anyway! if (!Name || Name[0] == '\0') @@ -415,6 +422,7 @@ Document* Application::newDocument(const char * Name, const char * UserName) _pActiveDoc->signalRedo.connect(boost::bind(&App::Application::slotRedoDocument, this, _1)); _pActiveDoc->signalRecomputedObject.connect(boost::bind(&App::Application::slotRecomputedObject, this, _1)); _pActiveDoc->signalRecomputed.connect(boost::bind(&App::Application::slotRecomputed, this, _1)); + _pActiveDoc->signalBeforeRecompute.connect(boost::bind(&App::Application::slotBeforeRecompute, this, _1)); _pActiveDoc->signalOpenTransaction.connect(boost::bind(&App::Application::slotOpenTransaction, this, _1, _2)); _pActiveDoc->signalCommitTransaction.connect(boost::bind(&App::Application::slotCommitTransaction, this, _1)); _pActiveDoc->signalAbortTransaction.connect(boost::bind(&App::Application::slotAbortTransaction, this, _1)); @@ -430,7 +438,7 @@ Document* Application::newDocument(const char * Name, const char * UserName) Py::Module("FreeCAD").setAttr(std::string("ActiveDocument"),active); } - signalNewDocument(*_pActiveDoc); + signalNewDocument(*_pActiveDoc,createView); // set the UserName after notifying all observers _pActiveDoc->Label.setValue(userName); @@ -466,6 +474,7 @@ bool Application::closeDocument(const char* name) void Application::closeAllDocuments(void) { + Base::FlagToggler flag(_isClosingAll); std::map::iterator pos; while((pos = DocMap.begin()) != DocMap.end()) closeDocument(pos->first.c_str()); @@ -524,7 +533,185 @@ std::string Application::getUniqueDocumentName(const char *Name) const } } -Document* Application::openDocument(const char * FileName) +int Application::addPendingDocument(const char *FileName, const char *objName, bool allowPartial) +{ + if(!_isRestoring) + return 0; + if(allowPartial && _allowPartial) + return -1; + assert(FileName && FileName[0]); + assert(objName && objName[0]); + auto ret = _pendingDocMap.emplace(FileName,std::set()); + ret.first->second.emplace(objName); + if(ret.second) { + _pendingDocs.push_back(ret.first->first.c_str()); + return 1; + } + return -1; +} + +bool Application::isRestoring() const { + return _isRestoring || Document::isAnyRestoring(); +} + +bool Application::isClosingAll() const { + return _isClosingAll; +} + +struct DocTiming { + FC_DURATION_DECLARE(d1); + FC_DURATION_DECLARE(d2); + DocTiming() { + FC_DURATION_INIT(d1); + FC_DURATION_INIT(d2); + } +}; + +class DocOpenGuard { +public: + bool &flag; + boost::signals2::signal &signal; + DocOpenGuard(bool &f, boost::signals2::signal &s) + :flag(f),signal(s) + { + flag = true; + } + ~DocOpenGuard() { + if(flag) { + flag = false; + signal(); + } + } +}; + +Document* Application::openDocument(const char * FileName, bool createView) { + std::vector filenames(1,FileName); + auto docs = openDocuments(filenames,0,0,0,createView); + if(docs.size()) + return docs.front(); + return 0; +} + +std::vector Application::openDocuments( + const std::vector &filenames, + const std::vector *pathes, + const std::vector *labels, + std::vector *errs, + bool createView) +{ + std::vector res(filenames.size(),nullptr); + if(filenames.empty()) + return res; + + if(errs) + errs->resize(filenames.size()); + + DocOpenGuard guard(_isRestoring,signalFinishOpenDocument); + _pendingDocs.clear(); + _pendingDocsReopen.clear(); + _pendingDocMap.clear(); + + signalStartOpenDocument(); + + ParameterGrp::handle hGrp = GetParameterGroupByPath("User parameter:BaseApp/Preferences/Document"); + _allowPartial = !hGrp->GetBool("NoPartialLoading",false); + + for(auto &name : filenames) + _pendingDocs.push_back(name.c_str()); + + std::deque > newDocs; + + FC_TIME_INIT(t); + + for(std::size_t count=0;;++count) { + const char *name = _pendingDocs.front(); + _pendingDocs.pop_front(); + bool isMainDoc = count objNames; + if(_allowPartial) { + auto it = _pendingDocMap.find(name); + if(it!=_pendingDocMap.end()) + objNames.swap(it->second); + } + FC_TIME_INIT(t1); + DocTiming timing; + const char *path = name; + const char *label = 0; + if(isMainDoc) { + if(pathes && pathes->size()>count) + path = (*pathes)[count].c_str(); + if(labels && labels->size()>count) + label = (*labels)[count].c_str(); + } + auto doc = openDocumentPrivate(path,name,label,isMainDoc,createView,objNames); + FC_DURATION_PLUS(timing.d1,t1); + if(doc) + newDocs.emplace_front(doc,timing); + if(isMainDoc) + res[count] = doc; + _objCount = -1; + }catch(const Base::Exception &e) { + if(!errs && isMainDoc) + throw; + if(errs && isMainDoc) + (*errs)[count] = e.what(); + else + Console().Error("Exception opening file: %s [%s]\n", name, e.what()); + }catch(const std::exception &e) { + if(!errs && isMainDoc) + throw; + if(errs && isMainDoc) + (*errs)[count] = e.what(); + else + Console().Error("Exception opening file: %s [%s]\n", name, e.what()); + }catch(...) { + if(errs) { + if(isMainDoc) + (*errs)[count] = "unknown error"; + } else { + _pendingDocs.clear(); + _pendingDocsReopen.clear(); + _pendingDocMap.clear(); + throw; + } + } + if(_pendingDocs.empty()) { + if(_pendingDocsReopen.empty()) + break; + _allowPartial = false; + _pendingDocs.swap(_pendingDocsReopen); + } + } + _pendingDocs.clear(); + _pendingDocsReopen.clear(); + _pendingDocMap.clear(); + + Base::SequencerLauncher seq("Postprocessing...", newDocs.size()); + for(auto &v : newDocs) { + FC_TIME_INIT(t1); + v.first->afterRestore(true); + FC_DURATION_PLUS(v.second.d2,t1); + seq.next(); + } + setActiveDocument(newDocs.back().first); + + for(auto &v : newDocs) { + FC_DURATION_LOG(v.second.d1, v.first->getName() << " restore"); + FC_DURATION_LOG(v.second.d2, v.first->getName() << " postprocess"); + } + FC_TIME_LOG(t,"total"); + + _isRestoring = false; + signalFinishOpenDocument(); + return res; +} + +Document* Application::openDocumentPrivate(const char * FileName, + const char *propFileName, const char *label, + bool isMainDoc, bool createView, + const std::set &objNames) { FileInfo File(FileName); @@ -539,23 +726,64 @@ Document* Application::openDocument(const char * FileName) for (std::map::iterator it = DocMap.begin(); it != DocMap.end(); ++it) { // get unique path separators std::string fi = FileInfo(it->second->FileName.getValue()).filePath(); - if (filepath == fi) { - std::stringstream str; - str << "The project '" << FileName << "' is already open!"; - throw Base::FileSystemError(str.str().c_str()); + if (filepath != fi) + continue; + if(it->second->testStatus(App::Document::PartialDoc) + || it->second->testStatus(App::Document::PartialRestore)) { + // Here means a document is already partially loaded, but the document + // is requested again, either partial or not. We must check if the + // document contains the required object + + if(isMainDoc) { + // Main document must be open fully, so close and reopen + closeDocument(it->first.c_str()); + break; + } + + if(_allowPartial) { + bool reopen = false; + for(auto &name : objNames) { + auto obj = it->second->getObject(name.c_str()); + if(!obj || obj->testStatus(App::PartialObject)) { + reopen = true; + break; + } + } + if(!reopen) + return 0; + } + auto &names = _pendingDocMap[FileName]; + names.clear(); + _pendingDocsReopen.push_back(FileName); + return 0; } + + if(!isMainDoc) + return 0; + std::stringstream str; + str << "The project '" << FileName << "' is already open!"; + throw Base::FileSystemError(str.str().c_str()); } + std::string name; + if(propFileName != FileName) { + FileInfo fi(propFileName); + name = fi.fileNamePure(); + }else + name = File.fileNamePure(); + // Use the same name for the internal and user name. // The file name is UTF-8 encoded which means that the internal name will be modified // to only contain valid ASCII characters but the user name will be kept. - Document* newDoc = newDocument(File.fileNamePure().c_str(), File.fileNamePure().c_str()); + if(!label) + label = name.c_str(); + Document* newDoc = newDocument(name.c_str(),label,isMainDoc && createView); - newDoc->FileName.setValue(File.filePath()); + newDoc->FileName.setValue(propFileName==FileName?File.filePath():propFileName); try { // read the document - newDoc->restore(); + newDoc->restore(File.filePath().c_str(),true,objNames); return newDoc; } // if the project file itself is corrupt then @@ -1235,6 +1463,11 @@ void Application::slotRecomputed(const Document& doc) this->signalRecomputed(doc); } +void Application::slotBeforeRecompute(const Document& doc) +{ + this->signalBeforeRecomputeDocument(doc); +} + void Application::slotOpenTransaction(const Document& d, string s) { this->signalOpenTransaction(d, s); diff --git a/src/App/Application.h b/src/App/Application.h index 255fb0c541..52e8c4e69d 100644 --- a/src/App/Application.h +++ b/src/App/Application.h @@ -86,13 +86,35 @@ public: * The second name is a UTF8 name of any kind. It's that name normally shown to * the user and stored in the App::Document::Name property. */ - App::Document* newDocument(const char * Name=0l, const char * UserName=0l); + App::Document* newDocument(const char * Name=0l, const char * UserName=0l, bool createView=true); /// Closes the document \a name and removes it from the application. bool closeDocument(const char* name); /// find a unique document name std::string getUniqueDocumentName(const char *Name) const; /// Open an existing document from a file - App::Document* openDocument(const char * FileName=0l); + App::Document* openDocument(const char * FileName=0l, bool createView=true); + /** Open multiple documents + * + * @param filenames: input file names + * @param pathes: optional input file path in case it is different from + * filenames (mainly used during recovery). + * @param labels: optional label assign to document (mainly used during recovery). + * @param errs: optional output error message corresponding to each input + * file name. If errs is given, this function will catch all + * Base::Exception and save the error message inside. Otherwise, it will + * throw on exception when opening the input files. + * @param createView: whether to signal Gui module to create view on restore. + * + * @return Return opened document object corresponding to each input file + * name, which maybe NULL if failed. + * + * This function will also open any external referenced files. + */ + std::vector openDocuments(const std::vector &filenames, + const std::vector *pathes=0, + const std::vector *labels=0, + std::vector *errs=0, + bool createView = true); /// Retrieve the active document App::Document* getActiveDocument(void) const; /// Retrieve a named document @@ -106,6 +128,14 @@ public: void setActiveDocument(const char *Name); /// close all documents (without saving) void closeAllDocuments(void); + /// Add pending document to open together with the current opening document + int addPendingDocument(const char *FileName, const char *objName, bool allowPartial); + /// Indicate whether the application is opening (restoring) some document + bool isRestoring() const; + /// Indicate the application is closing all document + bool isClosingAll() const; + //@} + /** @name Application-wide trandaction setting */ //@{ /** Setup a pending application-wide active transaction @@ -142,7 +172,7 @@ public: /** @name Signals of the Application */ //@{ /// signal on new Document - boost::signals2::signal signalNewDocument; + boost::signals2::signal signalNewDocument; /// signal on document getting deleted boost::signals2::signal signalDeleteDocument; /// signal on already deleted Document @@ -177,6 +207,10 @@ public: boost::signals2::signal signalCloseTransaction; /// signal on show hidden items boost::signals2::signal signalShowHidden; + /// signal on start opening document(s) + boost::signals2::signal signalStartOpenDocument; + /// signal on finished opening document(s) + boost::signals2::signal signalFinishOpenDocument; //@} @@ -202,6 +236,8 @@ public: boost::signals2::signal signalRelabelObject; /// signal on activated Object boost::signals2::signal signalActivatedObject; + /// signal before recomputed document + boost::signals2::signal signalBeforeRecomputeDocument; /// signal on recomputed document boost::signals2::signal signalRecomputed; /// signal on recomputed document object @@ -377,6 +413,7 @@ protected: void slotRedoDocument(const App::Document&); void slotRecomputedObject(const App::DocumentObject&); void slotRecomputed(const App::Document&); + void slotBeforeRecompute(const App::Document&); void slotOpenTransaction(const App::Document&, std::string); void slotCommitTransaction(const App::Document&); void slotAbortTransaction(const App::Document&); @@ -385,6 +422,10 @@ protected: void slotChangePropertyEditor(const App::Document&, const App::Property &); //@} + /// open single document only + App::Document* openDocumentPrivate(const char * FileName, const char *propFileName, + const char *label, bool isMainDoc, bool createView, const std::set &objNames); + /// Helper class for App::Document to signal on close/abort transaction class AppExport TransactionSignaller { public: @@ -439,6 +480,7 @@ private: static PyObject* sListDocuments (PyObject *self,PyObject *args); static PyObject* sAddDocObserver (PyObject *self,PyObject *args); static PyObject* sRemoveDocObserver (PyObject *self,PyObject *args); + static PyObject *sIsRestoring (PyObject *self,PyObject *args); static PyObject *sSetLogLevel (PyObject *self,PyObject *args); static PyObject *sGetLogLevel (PyObject *self,PyObject *args); @@ -446,9 +488,12 @@ private: static PyObject *sCheckLinkDepth (PyObject *self,PyObject *args); static PyObject *sGetLinksTo (PyObject *self,PyObject *args); + static PyObject *sGetDependentObjects(PyObject *self,PyObject *args); + static PyObject *sSetActiveTransaction (PyObject *self,PyObject *args); static PyObject *sGetActiveTransaction (PyObject *self,PyObject *args); static PyObject *sCloseActiveTransaction(PyObject *self,PyObject *args); + static PyObject *sCheckAbort(PyObject *self,PyObject *args); static PyMethodDef Methods[]; friend class ApplicationObserver; @@ -497,6 +542,13 @@ private: std::map &_mConfig; App::Document* _pActiveDoc; + std::deque _pendingDocs; + std::deque _pendingDocsReopen; + std::map > _pendingDocMap; + bool _isRestoring; + bool _allowPartial; + bool _isClosingAll; + // for estimate max link depth int _objCount; diff --git a/src/App/ApplicationPy.cpp b/src/App/ApplicationPy.cpp index 89cef1be04..f97ca05fc6 100644 --- a/src/App/ApplicationPy.cpp +++ b/src/App/ApplicationPy.cpp @@ -45,6 +45,7 @@ #include #include #include +#include //using Base::GetConsole; using namespace Base; @@ -132,8 +133,8 @@ PyMethodDef Application::Methods[] = { "Get a document by its name or raise an exception\n" "if there is no document with the given name."}, {"listDocuments", (PyCFunction) Application::sListDocuments, METH_VARARGS, - "listDocuments() -> list\n\n" - "Return a list of names of all documents."}, + "listDocuments(sort=False) -> list\n\n" + "Return a list of names of all documents, optionally sort in dependency order."}, {"addDocumentObserver", (PyCFunction) Application::sAddDocObserver, METH_VARARGS, "addDocumentObserver() -> None\n\n" "Add an observer to get notified about changes on documents."}, @@ -151,6 +152,12 @@ PyMethodDef Application::Methods[] = { "getLinksTo(obj,options=0,maxCount=0) -- return the objects linked to 'obj'\n\n" "options: 1: recursive, 2: check link array. Options can combine.\n" "maxCount: to limit the number of links returned\n"}, + {"getDependentObjects", (PyCFunction) Application::sGetDependentObjects, METH_VARARGS, + "getDependentObjects(obj|[obj,...], options=0)\n" + "Return a list of dependent objects including the given objects.\n\n" + "options: can have the following bit flags,\n" + " 1: to sort the list in topological order.\n" + " 2: to exclude dependency of Link type object."}, {"setActiveTransaction", (PyCFunction) Application::sSetActiveTransaction, METH_VARARGS, "setActiveTransaction(name, persist=False) -- setup active transaction with the given name\n\n" "name: the transaction name\n" @@ -164,6 +171,14 @@ PyMethodDef Application::Methods[] = { "getActiveTransaction() -> (name,id) return the current active transaction name and ID"}, {"closeActiveTransaction", (PyCFunction) Application::sCloseActiveTransaction, METH_VARARGS, "closeActiveTransaction(abort=False) -- commit or abort current active transaction"}, + {"isRestoring", (PyCFunction) Application::sIsRestoring, METH_VARARGS, + "isRestoring() -> Bool -- Test if the application is opening some document"}, + {"checkAbort", (PyCFunction) Application::sCheckAbort, METH_VARARGS, + "checkAbort() -- check for user abort in length operation.\n\n" + "This only works if there is an active sequencer (or ProgressIndicator in Python).\n" + "There is an active sequencer during document restore and recomputation. User may\n" + "abort the operation by pressing the ESC key. Once detected, this function will\n" + "trigger a BaseExceptionFreeCADAbort exception."}, {NULL, NULL, 0, NULL} /* Sentinel */ }; @@ -213,6 +228,12 @@ PyObject* Application::sLoadFile(PyObject * /*self*/, PyObject *args) } } +PyObject* Application::sIsRestoring(PyObject * /*self*/, PyObject *args) { + if (!PyArg_ParseTuple(args, "")) + return NULL; + return Py::new_reference_to(Py::Boolean(GetApplication().isRestoring())); +} + PyObject* Application::sOpenDocument(PyObject * /*self*/, PyObject *args) { char* Name; @@ -645,22 +666,26 @@ PyObject* Application::sGetHomePath(PyObject * /*self*/, PyObject *args) PyObject* Application::sListDocuments(PyObject * /*self*/, PyObject *args) { - if (!PyArg_ParseTuple(args, "")) // convert args: Python->C + PyObject *sort = Py_False; + if (!PyArg_ParseTuple(args, "|O",&sort)) // convert args: Python->C return NULL; // NULL triggers exception PY_TRY { PyObject *pDict = PyDict_New(); PyObject *pKey; Base::PyObjectBase* pValue; - for (std::map::const_iterator It = GetApplication().DocMap.begin(); - It != GetApplication().DocMap.end();++It) { + std::vector docs = GetApplication().getDocuments();; + if(PyObject_IsTrue(sort)) + docs = Document::getDependentDocuments(docs,true); + + for (auto doc : docs) { #if PY_MAJOR_VERSION >= 3 - pKey = PyUnicode_FromString(It->first.c_str()); + pKey = PyUnicode_FromString(doc->getName()); #else - pKey = PyString_FromString(It->first.c_str()); + pKey = PyString_FromString(doc->getName()); #endif // GetPyObject() increments - pValue = static_cast(It->second->getPyObject()); + pValue = static_cast(doc->getPyObject()); PyDict_SetItem(pDict, pKey, pValue); // now we can decrement again as PyDict_SetItem also has incremented pValue->DecRef(); @@ -819,6 +844,41 @@ PyObject *Application::sGetLinksTo(PyObject * /*self*/, PyObject *args) }PY_CATCH; } +PyObject *Application::sGetDependentObjects(PyObject * /*self*/, PyObject *args) +{ + PyObject *obj; + int options = 0; + if (!PyArg_ParseTuple(args, "O|i", &obj,&options)) + return 0; + + std::vector objs; + if(PySequence_Check(obj)) { + Py::Sequence seq(obj); + for(size_t i=0;i(seq[i].ptr())->getDocumentObjectPtr()); + } + }else if(!PyObject_TypeCheck(obj,&DocumentObjectPy::Type)) { + PyErr_SetString(PyExc_TypeError, + "Expect first argument to be either a document object or sequence of document objects"); + return 0; + }else + objs.push_back(static_cast(obj)->getDocumentObjectPtr()); + + PY_TRY { + auto ret = App::Document::getDependencyList(objs,options); + + Py::Tuple tuple(ret.size()); + for(size_t i=0;igetPyObject(),true)); + return Py::new_reference_to(tuple); + } PY_CATCH; +} + + PyObject *Application::sSetActiveTransaction(PyObject * /*self*/, PyObject *args) { char *name; @@ -862,3 +922,13 @@ PyObject *Application::sCloseActiveTransaction(PyObject * /*self*/, PyObject *ar } PY_CATCH; } +PyObject *Application::sCheckAbort(PyObject * /*self*/, PyObject *args) +{ + if (!PyArg_ParseTuple(args, "")) + return 0; + + PY_TRY { + Base::Sequencer().checkAbort(); + Py_Return; + }PY_CATCH +} diff --git a/src/App/Document.cpp b/src/App/Document.cpp index 1fa1631e5c..318a19f790 100644 --- a/src/App/Document.cpp +++ b/src/App/Document.cpp @@ -73,6 +73,7 @@ recompute path. Also, it enables more complicated dependencies beyond trees. #include #include #include +#include #ifdef USE_OLD_DAG #include @@ -107,6 +108,7 @@ recompute path. Also, it enables more complicated dependencies beyond trees. #include #include #include +#include #ifdef _MSC_VER #include @@ -156,14 +158,17 @@ typedef std::vector Path; namespace App { +static bool _IsRestoring; static bool _IsRelabeling; // Pimpl class struct DocumentP { // Array to preserve the creation order of created objects std::vector objectArray; + std::unordered_set touchedObjs; std::unordered_map objectMap; std::unordered_map objectIdMap; + std::unordered_map partialLoadObjects; long lastObjectId; DocumentObject* activeObject; Transaction *activeUndoTransaction; @@ -179,6 +184,8 @@ struct DocumentP std::map VertexObjectList; std::map vertexMap; #endif //USE_OLD_DAG + std::multimap > _RecomputeLog; DocumentP() { static std::random_device _RD; @@ -201,13 +208,44 @@ struct DocumentP UndoMaxStackSize = 20; } + void addRecomputeLog(const char *why, App::DocumentObject *obj) { + addRecomputeLog(new DocumentObjectExecReturn(why,obj)); + } + + void addRecomputeLog(const std::string &why, App::DocumentObject *obj) { + addRecomputeLog(new DocumentObjectExecReturn(why,obj)); + } + + void addRecomputeLog(DocumentObjectExecReturn *returnCode) { + if(!returnCode->Which) { + delete returnCode; + return; + } + _RecomputeLog.emplace(returnCode->Which, std::unique_ptr(returnCode)); + returnCode->Which->setStatus(ObjectStatus::Error,true); + } + + void clearRecomputeLog(const App::DocumentObject *obj=0) { + if(!obj) + _RecomputeLog.clear(); + else + _RecomputeLog.erase(obj); + } + + const char *findRecomputeLog(const App::DocumentObject *obj) { + auto range = _RecomputeLog.equal_range(obj); + if(range.first == range.second) + return 0; + return (--range.second)->second->Why.c_str(); + } + static void findAllPathsAt(const std::vector &all_nodes, size_t id, std::vector &all_paths, Path tmp); std::vector topologicalSort(const std::vector& objects) const; std::vector - partialTopologicalSort(const std::vector& objects) const; + static partialTopologicalSort(const std::vector& objects); }; } // namespace App @@ -903,7 +941,6 @@ bool Document::undo(int id) _commitTransaction(true); if (mUndoTransactions.empty()) return false; - // redo d->activeUndoTransaction = new Transaction(mUndoTransactions.back()->getID()); d->activeUndoTransaction->Name = mUndoTransactions.back()->Name; @@ -1137,7 +1174,6 @@ void Document::_commitTransaction(bool notify) FC_WARN("Cannot commit transaction while transacting"); return; } - if (d->activeUndoTransaction) { Application::TransactionSignaller signaller(false,true); int id = d->activeUndoTransaction->getID(); @@ -1582,7 +1618,8 @@ void Document::Save (Base::Writer &writer) const void Document::Restore(Base::XMLReader &reader) { int i,Cnt; - Base::ObjectStatusLocker restoreBit(Status::Restoring, this); + d->touchedObjs.clear(); + setStatus(Document::PartialDoc,false); reader.readElement("Document"); long scheme = reader.getAttributeAsInteger("SchemaVersion"); @@ -1662,9 +1699,50 @@ void Document::Restore(Base::XMLReader &reader) reader.readEndElement("Document"); } -void Document::exportObjects(const std::vector& obj, - std::ostream& out) -{ +struct DocExportStatus { + Document::ExportStatus status; + std::set objs; +}; + +static DocExportStatus _ExportStatus; + +// Exception-safe exporting status setter +class DocumentExporting { +public: + DocumentExporting(const std::vector &objs) { + _ExportStatus.status = Document::Exporting; + _ExportStatus.objs.insert(objs.begin(),objs.end()); + } + + ~DocumentExporting() { + _ExportStatus.status = Document::NotExporting; + _ExportStatus.objs.clear(); + } +}; + +// The current implementation choose to use a static variable for exporting +// status because we can be exporting multiple objects from multiple documents +// at the same time. I see no benefits in distinguish which documents are +// exporting, so just use a static variable for global status. But the +// implementation can easily be changed here if necessary. +Document::ExportStatus Document::isExporting(const App::DocumentObject *obj) const { + if(_ExportStatus.status!=Document::NotExporting && + (!obj || _ExportStatus.objs.find(obj)!=_ExportStatus.objs.end())) + return _ExportStatus.status; + return Document::NotExporting; +} + +void Document::exportObjects(const std::vector& obj, std::ostream& out) { + + DocumentExporting exporting(obj); + + if(FC_LOG_INSTANCE.isEnabled(FC_LOGLEVEL_LOG)) { + for(auto o : obj) { + if(o && o->getNameInDocument()) + FC_LOG("exporting " << o->getFullName()); + } + } + Base::ZipWriter writer(out); writer.putNextEntry("Document.xml"); writer.Stream() << "" << endl; @@ -1688,20 +1766,56 @@ void Document::exportObjects(const std::vector& obj, writer.writeFiles(); } +#define FC_ATTR_DEPENDENCIES "Dependencies" +#define FC_ELEMENT_OBJECT_DEPS "ObjectDeps" +#define FC_ATTR_DEP_COUNT "Count" +#define FC_ATTR_DEP_OBJ_NAME "Name" +#define FC_ATTR_DEP_COUNT "Count" +#define FC_ATTR_DEP_ALLOW_PARTIAL "AllowPartial" +#define FC_ELEMENT_OBJECT_DEP "Dep" void Document::writeObjects(const std::vector& obj, Base::Writer &writer) const { // writing the features types writer.incInd(); // indentation for 'Objects count' - writer.Stream() << writer.ind() << "" << endl; + writer.Stream() << writer.ind() << "" << endl; writer.incInd(); // indentation for 'Object type' + + if(!isExporting(0)) { + for(auto o : obj) { + const auto &outList = o->getOutList(DocumentObject::OutListNoHidden); + writer.Stream() << writer.ind() + << "<" FC_ELEMENT_OBJECT_DEPS " " FC_ATTR_DEP_OBJ_NAME "=\"" + << o->getNameInDocument() << "\" " FC_ATTR_DEP_COUNT "=\"" << outList.size(); + if(outList.empty()) { + writer.Stream() << "\"/>" << endl; + continue; + } + int partial = o->canLoadPartial(); + if(partial>0) + writer.Stream() << "\" " FC_ATTR_DEP_ALLOW_PARTIAL << "=\"" << partial; + writer.Stream() << "\">" << endl; + writer.incInd(); + for(auto dep : outList) { + auto name = dep?dep->getNameInDocument():""; + writer.Stream() << writer.ind() << "<" FC_ELEMENT_OBJECT_DEP " " + FC_ATTR_DEP_OBJ_NAME "=\"" << (name?name:"") << "\"/>" << endl; + } + writer.decInd(); + writer.Stream() << writer.ind() << "" << endl; + } + } + std::vector::const_iterator it; for (it = obj.begin(); it != obj.end(); ++it) { writer.Stream() << writer.ind() << "getTypeId().getName() << "\" " - << "name=\"" << (*it)->getNameInDocument() << "\" "; + << "name=\"" << (*it)->getExportName() << "\" " << "id=\"" << (*it)->getID() << "\" " << "ViewType=\"" << (*it)->getViewProviderNameStored() << "\" "; @@ -1725,7 +1839,7 @@ void Document::writeObjects(const std::vector& obj, writer.incInd(); // indentation for 'Object name' for (it = obj.begin(); it != obj.end(); ++it) { - writer.Stream() << writer.ind() << "getNameInDocument() << "\""; + writer.Stream() << writer.ind() << "getExportName() << "\""; if((*it)->hasExtensions()) writer.Stream() << " Extensions=\"True\""; @@ -1739,9 +1853,46 @@ void Document::writeObjects(const std::vector& obj, writer.decInd(); // indentation for 'Objects count' } +struct DepInfo { + std::unordered_set deps; + int canLoadPartial = 0; +}; + +static void _loadDeps(const std::string &name, + std::unordered_map &objs, + const std::unordered_map &deps) +{ + auto it = deps.find(name); + if(it == deps.end()) { + objs.emplace(name,true); + return; + } + if(it->second.canLoadPartial) { + if(it->second.canLoadPartial == 1) { + // canLoadPartial==1 means all its children will be created but not + // restored, i.e. exists as if newly created object, and therefore no + // need to load dependency of the children + for(auto &dep : it->second.deps) + objs.emplace(dep,false); + objs.emplace(name,true); + }else + objs.emplace(name,false); + return; + } + objs[name] = true; + // If cannot load partial, then recurse to load all children dependency + for(auto &dep : it->second.deps) { + auto it = objs.find(dep); + if(it!=objs.end() && it->second) + continue; + _loadDeps(dep,objs,deps); + } +} + std::vector Document::readObjects(Base::XMLReader& reader) { + d->touchedObjs.clear(); bool keepDigits = testStatus(Document::KeepTrailingDigits); setStatus(Document::KeepTrailingDigits, !reader.doNameMapping()); std::vector objs; @@ -1751,6 +1902,46 @@ Document::readObjects(Base::XMLReader& reader) reader.readElement("Objects"); int Cnt = reader.getAttributeAsInteger("Count"); + if(!reader.hasAttribute(FC_ATTR_DEPENDENCIES)) + d->partialLoadObjects.clear(); + else if(d->partialLoadObjects.size()) { + std::unordered_map deps; + for (int i=0 ;i objs; + objs.reserve(d->partialLoadObjects.size()); + for(auto &v : d->partialLoadObjects) + objs.push_back(v.first.c_str()); + for(auto &name : objs) + _loadDeps(name,d->partialLoadObjects,deps); + if(Cnt > (int)d->partialLoadObjects.size()) + setStatus(Document::PartialDoc,true); + else { + for(auto &v : d->partialLoadObjects) { + if(!v.second) { + setStatus(Document::PartialDoc,true); + break; + } + } + if(!testStatus(Document::PartialDoc)) + d->partialLoadObjects.clear(); + } + } + long lastId = 0; for (int i=0 ;ipartialLoadObjects.size()) { + auto it = d->partialLoadObjects.find(name); + if(it == d->partialLoadObjects.end()) + continue; + partial = !it->second; + } + if(!testStatus(Status::Importing) && reader.hasAttribute("id")) { // if not importing, then temporary reset lastObjectId and make the // following addObject() generate the correct id for this object. d->lastObjectId = reader.getAttributeAsInteger("id")-1; } + // To prevent duplicate name when export/import of objects from + // external documents, we append those external object name with + // @. Before importing (here means we are called by + // importObjects), we shall strip the postfix. What the caller + // (MergeDocument) sees is still the unstripped name mapped to a new + // internal name, and the rest of the link properties will be able to + // correctly unmap the names. + auto pos = name.find('@'); + std::string _obj_name; + const char *obj_name; + if(pos!=std::string::npos) { + _obj_name = name.substr(0,pos); + obj_name = _obj_name.c_str(); + }else + obj_name = name.c_str(); + try { // Use name from XML as is and do NOT remove trailing digits because // otherwise we may cause a dependency to itself @@ -1779,10 +1994,15 @@ Document::readObjects(Base::XMLReader& reader) reader.addName(name.c_str(), obj->getNameInDocument()); // restore touch/error status flags - if (reader.hasAttribute("Touched")) - obj->setStatus(ObjectStatus::Touch, reader.getAttributeAsInteger("Touched") != 0); - if (reader.hasAttribute("Invalid")) + if (reader.hasAttribute("Touched")) { + if(reader.getAttributeAsInteger("Touched") != 0) + d->touchedObjs.insert(obj); + } + if (reader.hasAttribute("Invalid")) { obj->setStatus(ObjectStatus::Error, reader.getAttributeAsInteger("Invalid") != 0); + if(obj->isError() && reader.hasAttribute("Error")) + d->addRecomputeLog(reader.getAttribute("Error"),obj); + } } } catch (const Base::Exception& e) { @@ -1803,10 +2023,10 @@ Document::readObjects(Base::XMLReader& reader) reader.readElement("Object"); std::string name = reader.getName(reader.getAttribute("name")); DocumentObject* pObj = getObject(name.c_str()); - - if (pObj) { // check if this feature has been registered + if (pObj && !pObj->testStatus(App::PartialObject)) { // check if this feature has been registered pObj->setStatus(ObjectStatus::Restore, true); try { + FC_TRACE("restoring " << pObj->getFullName()); pObj->Restore(reader); } // Try to continue only for certain exception types if not handled @@ -1838,10 +2058,20 @@ Document::readObjects(Base::XMLReader& reader) return objs; } +void Document::addRecomputeObject(DocumentObject *obj) { + if(testStatus(Status::Restoring) && obj) { + d->touchedObjs.insert(obj); + obj->touch(); + } +} + std::vector Document::importObjects(Base::XMLReader& reader) { - setStatus(Document::Restoring, true); + Base::FlagToggler<> flag(_IsRestoring,false); + Base::ObjectStatusLocker restoreBit(Status::Restoring, this); + Base::ObjectStatusLocker restoreBit2(Status::Importing, this); + ExpressionParser::ExpressionImporter expImporter(reader); reader.readElement("Document"); long scheme = reader.getAttributeAsInteger("SchemaVersion"); reader.DocumentSchema = scheme; @@ -1857,19 +2087,25 @@ Document::importObjects(Base::XMLReader& reader) } std::vector objs = readObjects(reader); + for(auto o : objs) { + if(o && o->getNameInDocument()) { + o->setStatus(App::ObjImporting,true); + FC_LOG("importing " << o->getFullName()); + } + } reader.readEndElement("Document"); signalImportObjects(objs, reader); + afterRestore(objs,true); - // reset all touched - for (std::vector::iterator it= objs.begin();it!=objs.end();++it) { - (*it)->onDocumentRestored(); - (*it)->ExpressionEngine.onDocumentRestored(); - (*it)->purgeTouched(); + signalFinishImportObjects(objs); + + for(auto o : objs) { + if(o && o->getNameInDocument()) + o->setStatus(App::ObjImporting,false); } - setStatus(Document::Restoring, false); return objs; } @@ -1891,9 +2127,29 @@ unsigned int Document::getMemSize (void) const return size; } -bool Document::saveAs(const char* file) +static std::string checkFileName(const char *file) { + std::string fn(file); + + // Append extension if missing. This option is added for security reason, so + // that the user won't accidently overwrite other file that may be critical. + if(App::GetApplication().GetParameterGroupByPath + ("User parameter:BaseApp/Preferences/Document")->GetBool("CheckExtension",true)) + { + const char *ext = strrchr(file,'.'); + if(!ext || !boost::iequals(ext+1,"fcstd")) { + if(ext && ext[1] == 0) + fn += "fcstd"; + else + fn += ".fcstd"; + } + } + return fn; +} + +bool Document::saveAs(const char* _file) { - Base::FileInfo fi(file); + std::string file = checkFileName(_file); + Base::FileInfo fi(file.c_str()); if (this->FileName.getStrValue() != file) { this->FileName.setValue(file); this->Label.setValue(fi.fileNamePure()); @@ -1903,10 +2159,11 @@ bool Document::saveAs(const char* file) return save(); } -bool Document::saveCopy(const char* file) const +bool Document::saveCopy(const char* _file) const { + std::string file = checkFileName(_file); if (this->FileName.getStrValue() != file) { - bool result = saveToFile(file); + bool result = saveToFile(file.c_str()); return result; } return false; @@ -1915,6 +2172,14 @@ bool Document::saveCopy(const char* file) const // Save the document under the name it has been opened bool Document::save (void) { + if(testStatus(Document::PartialDoc)) { + FC_ERR("Partial loaded document '" << Label.getValue() << "' cannot be saved"); + // TODO We don't make this a fatal error and return 'true' to make it possible to + // save other documents that depends on this partial opened document. We need better + // handling to avoid touching partial documents. + return true; + } + if (*(FileName.getValue()) != '\0') { // Save the name of the tip object in order to handle in Restore() if (Tip.getValue()) { @@ -2060,51 +2325,68 @@ bool Document::saveToFile(const char* filename) const return true; } +bool Document::isAnyRestoring() { + return _IsRestoring; +} + // Open the document -void Document::restore (void) +void Document::restore (const char *filename, + bool delaySignal, const std::set &objNames) { - // clean up if the document is not empty - // !TODO mind exceptions while restoring! clearUndos(); - // first notify the objects to being deleted and then delete them in a second loop (#0002521) - // FIXME: To delete every object individually is inefficient. Add a new signal 'signalClear' - // and then clear everything in one go. - for (std::vector::iterator obj = d->objectArray.begin(); obj != d->objectArray.end(); ++obj) { - signalDeletedObject(*(*obj)); - signalTransactionRemove(*(*obj), 0); - } - for (std::vector::iterator obj = d->objectArray.begin(); obj != d->objectArray.end(); ++obj) { - (*obj)->setStatus(ObjectStatus::Destroy, true); - delete *obj; + d->activeObject = 0; + + if(d->objectArray.size()) { + GetApplication().signalDeleteDocument(*this); + d->objectArray.clear(); + for(auto &v : d->objectMap) { + v.second->setStatus(ObjectStatus::Destroy, true); + delete(v.second); + } + d->objectMap.clear(); + d->objectIdMap.clear(); + GetApplication().signalNewDocument(*this,false); } + + Base::FlagToggler<> flag(_IsRestoring,false); + + setStatus(Document::PartialDoc,false); + + d->clearRecomputeLog(); d->objectArray.clear(); d->objectMap.clear(); d->objectIdMap.clear(); d->lastObjectId = 0; - Base::FileInfo fi(FileName.getValue()); + if(!filename) + filename = FileName.getValue(); + Base::FileInfo fi(filename); Base::ifstream file(fi, std::ios::in | std::ios::binary); std::streambuf* buf = file.rdbuf(); std::streamoff size = buf->pubseekoff(0, std::ios::end, std::ios::in); buf->pubseekoff(0, std::ios::beg, std::ios::in); if (size < 22) // an empty zip archive has 22 bytes - throw Base::FileException("Invalid project file",FileName.getValue()); + throw Base::FileException("Invalid project file",filename); zipios::ZipInputStream zipstream(file); - Base::XMLReader reader(FileName.getValue(), zipstream); + Base::XMLReader reader(filename, zipstream); if (!reader.isValid()) - throw Base::FileException("Error reading compression file",FileName.getValue()); + throw Base::FileException("Error reading compression file",filename); GetApplication().signalStartRestoreDocument(*this); setStatus(Document::Restoring, true); + d->partialLoadObjects.clear(); + for(auto &name : objNames) + d->partialLoadObjects.emplace(name,true); try { Document::Restore(reader); } catch (const Base::Exception& e) { Base::Console().Error("Invalid Document.xml: %s\n", e.what()); } + d->partialLoadObjects.clear(); // Special handling for Gui document, the view representations must already // exist, what is done in Restore(). @@ -2113,26 +2395,121 @@ void Document::restore (void) signalRestoreDocument(reader); reader.readFiles(zipstream); - // reset all touched - for (std::map::iterator It= d->objectMap.begin();It!=d->objectMap.end();++It) { - It->second->connectRelabelSignals(); - try { - It->second->onDocumentRestored(); - It->second->ExpressionEngine.onDocumentRestored(); - } - catch (const Base::Exception& e) { - Base::Console().Error("Error in %s: %s\n", It->second->Label.getValue(), e.what()); - } - It->second->purgeTouched(); - } - - GetApplication().signalFinishRestoreDocument(*this); - setStatus(Document::Restoring, false); - if (reader.testStatus(Base::XMLReader::ReaderStatus::PartialRestore)) { setStatus(Document::PartialRestore, true); Base::Console().Error("There were errors while loading the file. Some data might have been modified or not recovered at all. Look above for more specific information about the objects involved.\n"); } + + if(!delaySignal) + afterRestore(true); +} + +void Document::afterRestore(bool checkPartial) { + Base::FlagToggler<> flag(_IsRestoring,false); + if(!afterRestore(d->objectArray,checkPartial)) { + FC_WARN("Reload partial document " << getName()); + restore(); + return; + } + GetApplication().signalFinishRestoreDocument(*this); + setStatus(Document::Restoring, false); +} + +bool Document::afterRestore(const std::vector &objArray, bool checkPartial) +{ + checkPartial = checkPartial && testStatus(Document::PartialDoc); + if(checkPartial && d->touchedObjs.size()) + return false; + + // some link type property cannot restore link information until other + // objects has been restored. For example, PropertyExpressionEngine and + // PropertySheet with expression containing label reference. So we add the + // Property::afterRestore() interface to let them sort it out. Note, this + // API is not called in object dedpenency order, because the order + // information is not ready yet. + std::map > propMap; + for(auto obj : objArray) { + auto &props = propMap[obj]; + obj->getPropertyList(props); + for(auto prop : props) { + try { + prop->afterRestore(); + } catch (const Base::Exception& e) { + FC_ERR("Failed to restore " << obj->getFullName() + << '.' << prop->getName() << ": " << e.what()); + } + } + } + + if(checkPartial && d->touchedObjs.size()) { + // partial document touched, signal full reload + return false; + } + + std::set objSet(objArray.begin(),objArray.end()); + auto objs = getDependencyList(objArray.empty()?d->objectArray:objArray,DepSort); + for (auto obj : objs) { + if(objSet.find(obj)==objSet.end()) + continue; + try { + for(auto prop : propMap[obj]) + prop->onContainerRestored(); + bool touched = false; + auto returnCode = obj->ExpressionEngine.execute( + PropertyExpressionEngine::ExecuteOnRestore,&touched); + if(returnCode!=DocumentObject::StdReturn) { + FC_ERR("Expression engine failed to restore " << obj->getFullName() << ": " << returnCode->Why); + d->addRecomputeLog(returnCode); + } + obj->onDocumentRestored(); + if(touched) + d->touchedObjs.insert(obj); + } + catch (const Base::Exception& e) { + d->addRecomputeLog(e.what(),obj); + FC_ERR("Failed to restore " << obj->getFullName() << ": " << e.what()); + } + catch (std::exception &e) { + d->addRecomputeLog(e.what(),obj); + FC_ERR("Failed to restore " << obj->getFullName() << ": " << e.what()); + } + catch (...) { + d->addRecomputeLog("Unknown exception on restore",obj); + FC_ERR("Failed to restore " << obj->getFullName() << ": " << "unknown exception"); + } + if(obj->isValid()) { + auto &props = propMap[obj]; + props.clear(); + // refresh properties in case the object changes its property list + obj->getPropertyList(props); + for(auto prop : props) { + auto link = Base::freecad_dynamic_cast(prop); + int res; + std::string errMsg; + if(link && (res=link->checkRestore(&errMsg))) { + d->touchedObjs.insert(obj); + if(res==1) + FC_WARN(obj->getFullName() << '.' << prop->getName() << ": " << errMsg); + else { + FC_ERR(obj->getFullName() << '.' << prop->getName() << ": " << errMsg); + d->addRecomputeLog(errMsg,obj); + setStatus(Document::PartialRestore, true); + } + } + } + } + + if(checkPartial && d->touchedObjs.size()) { + // partial document touched, signal full reload + return false; + } else if(!d->touchedObjs.count(obj)) + obj->purgeTouched(); + + signalFinishRestoreObject(*obj); + } + + d->touchedObjs.clear(); + return true; } bool Document::isSaved() const @@ -2284,151 +2661,218 @@ std::vector Document::getInList(const DocumentObject* me) return result; } -#ifdef USE_OLD_DAG -namespace boost { -// recursive helper function to get all dependencies -void out_edges_recursive(const Vertex& v, const DependencyList& g, std::set& out) +// This function unifies the old _rebuildDependencyList() and +// getDependencyList(). The algorithm basically obtains the object dependency +// by recrusivly visiting the OutList of each object in the given object array. +// It makes sure to call getOutList() of each object once and only once, which +// makes it much more efficient than calling getRecursiveOutList() on each +// individual object. +// +// The problem with the original algorithm is that, it assumes the objects +// inside any OutList are all within the given object array, so it does not +// recursively call getOutList() on those dependent objects inside. This +// assumption is broken by the introduction of PropertyXLink which can link to +// external object. +// +static void _buildDependencyList(const std::vector &objectArray, + int options, std::vector *depObjs, + DependencyList *depList, std::map *objectMap, + bool *touchCheck = 0) { - DependencyList::out_edge_iterator j, jend; - for (boost::tie(j, jend) = boost::out_edges(v, g); j != jend; ++j) { - Vertex n = boost::target(*j, g); - std::pair::iterator, bool> i = out.insert(n); - if (i.second) - out_edges_recursive(n, g, out); - } -} -} + std::map > outLists; + std::deque objs; -std::vector -Document::getDependencyList(const std::vector& objs) const -{ - DependencyList DepList; - std::map ObjectMap; - std::map VertexMap; + if(objectMap) objectMap->clear(); + if(depList) depList->clear(); - // Filling up the adjacency List - for (std::vector::const_iterator it = d->objectArray.begin(); it != d->objectArray.end();++it) { - // add the object as Vertex and remember the index - Vertex v = add_vertex(DepList); - ObjectMap[*it] = v; - VertexMap[v] = *it; - } + int op = (options & Document::DepNoXLinked)?DocumentObject::OutListNoXLinked:0; + for (auto obj : objectArray) { + objs.push_back(obj); + while(objs.size()) { + auto obj = objs.front(); + objs.pop_front(); + if(!obj || !obj->getNameInDocument()) + continue; - for (std::vector::const_iterator it = d->objectArray.begin(); it != d->objectArray.end();++it) { - std::vector outList = (*it)->getOutList(); - for (std::vector::const_iterator jt = outList.begin(); jt != outList.end();++jt) { - if (*jt) { - std::map::const_iterator i = ObjectMap.find(*jt); + auto it = outLists.find(obj); + if(it!=outLists.end()) + continue; - if (i == ObjectMap.end()) { - Vertex v = add_vertex(DepList); - - ObjectMap[*jt] = v; - VertexMap[v] = *jt; + if(touchCheck) { + if(obj->isTouched() || obj->mustExecute()) { + // early termination on touch check + *touchCheck = true; + return; } } + if(depObjs) depObjs->push_back(obj); + if(objectMap && depList) + (*objectMap)[obj] = add_vertex(*depList); + + auto &outList = outLists[obj]; + outList = obj->getOutList(op); + objs.insert(objs.end(),outList.begin(),outList.end()); } } - // add the edges - for (std::vector::const_iterator it = d->objectArray.begin(); it != d->objectArray.end();++it) { - std::vector outList = (*it)->getOutList(); - for (std::vector::const_iterator jt = outList.begin(); jt != outList.end();++jt) { - if (*jt) { - add_edge(ObjectMap[*it],ObjectMap[*jt],DepList); + if(objectMap && depList) { + for (const auto &v : outLists) { + for(auto obj : v.second) { + if(obj && obj->getNameInDocument()) + add_edge((*objectMap)[v.first],(*objectMap)[obj],*depList); } } } +} + +std::vector Document::getDependencyList( + const std::vector& objectArray, int options) +{ + std::vector ret; + if(!(options & DepSort)) { + _buildDependencyList(objectArray,options,&ret,0,0); + return ret; + } + + DependencyList depList; + std::map objectMap; + std::map vertexMap; + + _buildDependencyList(objectArray,options,0,&depList,&objectMap); + + for(auto &v : objectMap) + vertexMap[v.second] = v.first; + + std::list make_order; + try { + boost::topological_sort(depList, std::front_inserter(make_order)); + } catch (const std::exception& e) { + if(options & DepNoCycle) { + // Use boost::strong_components to find cycles. It groups strongly + // connected vertices as components, and therefore each component + // forms a cycle. + std::vector c(vertexMap.size()); + std::map > components; + boost::strong_components(depList,boost::make_iterator_property_map( + c.begin(),boost::get(boost::vertex_index,depList),c[0])); + for(size_t i=0;isecond->getOutList()) { + if(obj == it->second) { + ss << std::endl << it->second->getFullName() << std::endl; + break; + } + } + continue; + } + // For components with more than one member, they form a loop together + for(size_t i=0;isecond->getFullName() << ", "; + } + ss << std::endl; + } + FC_ERR(ss.str()); + FC_THROWM(Base::RuntimeError, e.what()); + } + FC_ERR(e.what()); + ret = DocumentP::partialTopologicalSort(objectArray); + std::reverse(ret.begin(),ret.end()); + return ret; + } + + for (std::list::reverse_iterator i = make_order.rbegin();i != make_order.rend(); ++i) + ret.push_back(vertexMap[*i]); + return ret; +} + +std::vector Document::getDependentDocuments(bool sort) { + return getDependentDocuments({this},sort); +} + +std::vector Document::getDependentDocuments( + std::vector pending, bool sort) +{ + DependencyList depList; + std::map docMap; + std::map vertexMap; + + std::vector ret; + if(pending.empty()) + return ret; + + auto outLists = PropertyXLink::getDocumentOutList(); + std::set docs; + docs.insert(pending.begin(),pending.end()); + if(sort) { + for(auto doc : pending) + docMap[doc] = add_vertex(depList); + } + while(pending.size()) { + auto doc = pending.back(); + pending.pop_back(); + + auto it = outLists.find(doc); + if(it == outLists.end()) + continue; + + auto &vertex = docMap[doc]; + for(auto depDoc : it->second) { + if(docs.insert(depDoc).second) { + pending.push_back(depDoc); + if(sort) + docMap[depDoc] = add_vertex(depList); + } + add_edge(vertex,docMap[depDoc],depList); + } + } + + if(!sort) { + ret.insert(ret.end(),docs.begin(),docs.end()); + return ret; + } std::list make_order; - DependencyList::out_edge_iterator j, jend; - try { - // this sort gives the execute - boost::topological_sort(DepList, std::front_inserter(make_order)); - } - catch (const std::exception& e) { - std::stringstream ss; - ss << "Gathering all dependencies failed, probably due to circular dependencies. Error: "; - ss << e.what(); - throw Base::BadGraphError(ss.str().c_str()); + boost::topological_sort(depList, std::front_inserter(make_order)); + } catch (const std::exception& e) { + std::string msg("Document::getDependentDocuments: "); + msg += e.what(); + throw Base::RuntimeError(msg); } - std::set out; - for (std::vector::const_iterator it = objs.begin(); it != objs.end(); ++it) { - std::map::iterator jt = ObjectMap.find(*it); - // ok, object is part of this graph - if (jt != ObjectMap.end()) { - out.insert(jt->second); - out_edges_recursive(jt->second, DepList, out); - } - } - - std::vector ary; - ary.reserve(out.size()); - for (std::set::iterator it = out.begin(); it != out.end(); ++it) - ary.push_back(VertexMap[*it]); - return ary; + for(auto &v : docMap) + vertexMap[v.second] = v.first; + for (auto rIt=make_order.rbegin(); rIt!=make_order.rend(); ++rIt) + ret.push_back(vertexMap[*rIt]); + return ret; } -#endif -void Document::_rebuildDependencyList(void) +void Document::_rebuildDependencyList(const std::vector &objs) { #ifdef USE_OLD_DAG - d->VertexObjectList.clear(); - d->DepList.clear(); - // Filling up the adjacency List - for (std::map::const_iterator It = d->objectMap.begin(); It != d->objectMap.end();++It) { - // add the object as Vertex and remember the index - d->VertexObjectList[It->second] = add_vertex(d->DepList); - } - - // add the edges - for (std::map::const_iterator It = d->objectMap.begin(); It != d->objectMap.end();++It) { - std::vector OutList = It->second->getOutList(); - for (std::vector::const_iterator It2=OutList.begin();It2!=OutList.end();++It2) { - if (*It2) { - std::map::iterator i = d->VertexObjectList.find(*It2); - - if (i == d->VertexObjectList.end()) - d->VertexObjectList[*It2] = add_vertex(d->DepList); - } - } - } - - // add the edges - for (std::map::const_iterator It = d->objectMap.begin(); It != d->objectMap.end();++It) { - std::vector OutList = It->second->getOutList(); - for (std::vector::const_iterator It2=OutList.begin();It2!=OutList.end();++It2) { - if (*It2) - add_edge(d->VertexObjectList[It->second],d->VertexObjectList[*It2],d->DepList); - } - } + _buildDependencyList(objs.empty()?d->objectArray:objs,false,0,&d->DepList,&d->VertexObjectList); +#else + (void)objs; #endif } -#ifndef USE_OLD_DAG -std::vector Document::getDependencyList(const std::vector& objs) const -{ - std::vector dep; - for (auto obj : objs){ - if(!obj) - continue; - std::vector objDep = obj->getOutListRecursive(); - dep.insert(dep.end(), objDep.begin(), objDep.end()); - dep.push_back(obj); - } - - // remove duplicate entries and resize the vector - std::sort(dep.begin(), dep.end()); - auto newEnd = std::unique(dep.begin(), dep.end()); - dep.resize(std::distance(dep.begin(), newEnd)); - - return dep; -} -#endif // USE_OLD_DAG - - /** * @brief Signal that object identifiers, typically a property or document object has been renamed. * @@ -2454,7 +2898,7 @@ void Document::renameObjectIdentifiers(const std::map &objs, bool force) { if (testStatus(Document::Recomputing)) { // this is clearly a bug in the calling instance @@ -2465,19 +2909,16 @@ int Document::recompute() // The 'SkipRecompute' flag can be (tmp.) set to avoid too many // time expensive recomputes - bool skip = testStatus(Document::SkipRecompute); - if (skip) + if(!force && testStatus(Document::SkipRecompute)) return 0; Base::ObjectStatusLocker exe(Document::Recomputing, this); // delete recompute log - for (std::vector::iterator it=_RecomputeLog.begin();it!=_RecomputeLog.end();++it) - delete *it; - _RecomputeLog.clear(); + d->clearRecomputeLog(); // updates the dependency graph - _rebuildDependencyList(); + _rebuildDependencyList(objs); std::list make_order; DependencyList::out_edge_iterator j, jend; @@ -2503,7 +2944,13 @@ int Document::recompute() for (std::list::reverse_iterator i = make_order.rbegin();i != make_order.rend(); ++i) { DocumentObject* Cur = d->vertexMap[*i]; - if (!Cur || !isIn(Cur)) continue; + // Because of PropertyXLink, we should account for external objects + // TODO: make sure it is safe to rely on getNameInDocument() to check if + // object is in the document. If it crashes, then we should fix the code + // to properly nullify getNameInDocument(), rather than revert back to + // the inefficient isIn() + // if (!Cur || !isIn(Cur)) continue; + if (!Cur || !Cur->getNameInDocument()) continue; #ifdef FC_LOGFEATUREUPDATE std::clog << Cur->getNameInDocument() << " dep on:" ; #endif @@ -2571,7 +3018,9 @@ int Document::recompute() // reset all touched for (std::map::iterator it = d->vertexMap.begin(); it != d->vertexMap.end(); ++it) { - if ((it->second) && isIn(it->second)) + // TODO: check the TODO comments above for details + // if ((it->second) && isIn(it->second)) + if ((it->second) && it->second->getNameInDocument()) it->second->purgeTouched(); } d->vertexMap.clear(); @@ -2583,71 +3032,131 @@ int Document::recompute() #else //ifdef USE_OLD_DAG -int Document::recompute() +int Document::recompute(const std::vector &objs, bool force, bool *hasError, int options) { + int objectCount = 0; + + if (testStatus(Document::PartialDoc)) { + if(mustExecute()) + FC_WARN("Please reload partial document '" << Label.getValue() << "' for recomputation."); + return 0; + } if (testStatus(Document::Recomputing)) { // this is clearly a bug in the calling instance throw Base::RuntimeError("Nested recomputes of a document are not allowed"); } - - int objectCount = 0; - // The 'SkipRecompute' flag can be (tmp.) set to avoid too many // time expensive recomputes - bool skip = testStatus(Document::SkipRecompute); - if (skip) + if(!force && testStatus(Document::SkipRecompute)) { + signalSkipRecompute(*this,objs); return 0; - - Base::ObjectStatusLocker exe(Document::Recomputing, this); + } // delete recompute log - for (auto LogEntry: _RecomputeLog) - delete LogEntry; - _RecomputeLog.clear(); + d->clearRecomputeLog(); //do we have anything to do? if(d->objectMap.empty()) return 0; - // get the sorted vector of all objects in the document and go though it from the end - vector topoSortedObjects = topologicalSort(); + Base::ObjectStatusLocker exe(Document::Recomputing, this); + signalBeforeRecompute(*this); - if (topoSortedObjects.size() != d->objectArray.size()){ +#if 0 + ////////////////////////////////////////////////////////////////////////// + // Comment by Realthunder: + // the topologicalSrot() below cannot handle partial recompute, haven't got + // time to figure out the code yet, simply use back boost::topological_sort + // for now, that is, rely on getDependencyList() to do the sorting. The + // downside is, it didn't take advantage of the ready built InList, nor will + // it report for cyclic dependency. + ////////////////////////////////////////////////////////////////////////// + + // get the sorted vector of all dependent objects and go though it from the end + auto depObjs = getDependencyList(objs.empty()?d->objectArray:objs); + vector topoSortedObjects = topologicalSort(depObjs); + if (topoSortedObjects.size() != depObjs.size()){ cerr << "App::Document::recompute(): cyclic dependency detected" << endl; - topoSortedObjects = d->partialTopologicalSort(d->objectArray); + topoSortedObjects = d->partialTopologicalSort(depObjs); } + std::reverse(topoSortedObjects.begin(),topoSortedObjects.end()); +#else + auto topoSortedObjects = getDependencyList(objs.empty()?d->objectArray:objs,DepSort|options); +#endif + for(auto obj : topoSortedObjects) + obj->setStatus(ObjectStatus::PendingRecompute,true); - for (auto objIt = topoSortedObjects.rbegin(); objIt != topoSortedObjects.rend(); ++objIt){ - // ask the object if it should be recomputed - bool doRecompute = false; - if ((*objIt)->mustRecompute()) { - doRecompute = true; - objectCount++; - if (_recomputeFeature(*objIt)) { - // if something happened break execution of recompute - return -1; + std::set filter; + size_t idx = 0; + try { + // maximum two passes to allow some form of dependency inversion + for(int passes=0; passes<2 && idxgetNameInDocument() || filter.find(obj)!=filter.end()) + continue; + // ask the object if it should be recomputed + bool doRecompute = false; + if (obj->mustRecompute()) { + doRecompute = true; + ++objectCount; + int res = _recomputeFeature(obj); + if(res) { + if(hasError) + *hasError = true; + if(res < 0) { + passes = 2; + break; + } + // if something happened filter all object in its + // inListRecursive from the queue then proceed + obj->getInListEx(filter,true); + filter.insert(obj); + continue; + } + } + if(obj->isTouched() || doRecompute) { + signalRecomputedObject(*obj); + obj->purgeTouched(); + // set all dependent object touched to force recompute + for (auto inObjIt : obj->getInList()) + inObjIt->enforceRecompute(); + } + } + // check if all objects are recomputed but still thouched + for (size_t i=0;isetStatus(ObjectStatus::Recompute2,false); + if(!filter.count(obj) && obj->isTouched()) { + if(passes>0) + FC_ERR(obj->getFullName() << " still touched after recompute"); + else{ + FC_LOG(obj->getFullName() << " still touched after recompute"); + if(idx>=topoSortedObjects.size()) { + // let's start the next pass on the first touched object + idx = i; + } + obj->setStatus(ObjectStatus::Recompute2,true); + } + } } - - signalRecomputedObject(*(*objIt)); - } - - if ((*objIt)->isTouched() || doRecompute) { - (*objIt)->purgeTouched(); - // force recompute of all dependent objects - for (auto inObjIt : (*objIt)->getInList()) - inObjIt->enforceRecompute(); } + }catch(Base::Exception &e) { + e.ReportException(); } - // check if all objects are recalculated which were touched - for (auto objectIt : d->objectArray) { - if (objectIt->isTouched()) { - Base::Console().Warning("Document::recompute(): %s still touched after recompute\n", - objectIt->getNameInDocument()); - } + for(auto obj : topoSortedObjects) { + if(!obj->getNameInDocument()) + continue; + obj->setStatus(ObjectStatus::PendingRecompute,false); + obj->setStatus(ObjectStatus::Recompute2,false); + if(obj->testStatus(ObjectStatus::PendingRemove)) + obj->getDocument()->removeObject(obj->getNameInDocument()); } - signalRecomputed(*this); + signalRecomputed(*this,topoSortedObjects); return objectCount; } @@ -2662,7 +3171,8 @@ int Document::recompute() An alternative to this method might be: https://en.wikipedia.org/wiki/Tarjan%E2%80%99s_strongly_connected_components_algorithm */ -std::vector DocumentP::partialTopologicalSort(const std::vector& objects) const +std::vector DocumentP::partialTopologicalSort( + const std::vector& objects) { vector < App::DocumentObject* > ret; ret.reserve(objects.size()); @@ -2771,6 +3281,10 @@ std::vector DocumentP::topologicalSort(const std::vector countMap; for (auto objectIt : objects) { + // We now support externally linked objects + // if(!obj->getNameInDocument() || obj->getDocument()!=this) + if(!objectIt->getNameInDocument()) + continue; //we need inlist with unique entries auto in = objectIt->getInList(); std::sort(in.begin(), in.end()); @@ -2784,7 +3298,7 @@ std::vector DocumentP::topologicalSort(const std::vector Document::topologicalSort() const const char * Document::getErrorDescription(const App::DocumentObject*Obj) const { - for (std::vector::const_iterator it=_RecomputeLog.begin();it!=_RecomputeLog.end();++it) - if ((*it)->Which == Obj) - return (*it)->Why.c_str(); - return 0; + return d->findRecomputeLog(Obj); } // call the recompute of the Feature and handle the exceptions and errors. -bool Document::_recomputeFeature(DocumentObject* Feat) +int Document::_recomputeFeature(DocumentObject* Feat) { -#ifdef FC_LOGFEATUREUPDATE - std::clog << "Solv: Executing Feature: " << Feat->getNameInDocument() << std::endl;; -#endif + FC_LOG("Recomputing " << Feat->getFullName()); DocumentObjectExecReturn *returnCode = 0; try { - returnCode = Feat->ExpressionEngine.execute(); - if (returnCode != DocumentObject::StdReturn) { - returnCode->Which = Feat; - _RecomputeLog.push_back(returnCode); - #ifdef FC_DEBUG - Base::Console().Error("Error in feature: %s\n%s\n",Feat->getNameInDocument(),returnCode->Why.c_str()); - #endif - Feat->setError(); - return true; + returnCode = Feat->ExpressionEngine.execute(PropertyExpressionEngine::ExecuteNonOutput); + if (returnCode == DocumentObject::StdReturn) { + returnCode = Feat->recompute(); + if(returnCode == DocumentObject::StdReturn) + returnCode = Feat->ExpressionEngine.execute(PropertyExpressionEngine::ExecuteOutput); } - - returnCode = Feat->recompute(); } catch(Base::AbortException &e){ e.ReportException(); - _RecomputeLog.push_back(new DocumentObjectExecReturn("User abort",Feat)); - Feat->setError(); - return true; + d->addRecomputeLog("User abort",Feat); + return -1; } catch (const Base::MemoryException& e) { - Base::Console().Error("Memory exception in feature '%s' thrown: %s\n",Feat->getNameInDocument(),e.what()); - _RecomputeLog.push_back(new DocumentObjectExecReturn("Out of memory exception",Feat)); - Feat->setError(); - return true; + FC_ERR("Memory exception in " << Feat->getFullName() << " thrown: " << e.what()); + d->addRecomputeLog("Out of memory exception",Feat); + return 1; } catch (Base::Exception &e) { e.ReportException(); - _RecomputeLog.push_back(new DocumentObjectExecReturn(e.what(),Feat)); - Feat->setError(); - return false; + d->addRecomputeLog(e.what(),Feat); + return 1; } catch (std::exception &e) { - Base::Console().Warning("exception in Feature \"%s\" thrown: %s\n",Feat->getNameInDocument(),e.what()); - _RecomputeLog.push_back(new DocumentObjectExecReturn(e.what(),Feat)); - Feat->setError(); - return false; + FC_ERR("exception in " << Feat->getFullName() << " thrown: " << e.what()); + d->addRecomputeLog(e.what(),Feat); + return 1; } #ifndef FC_DEBUG catch (...) { - Base::Console().Error("App::Document::_RecomputeFeature(): Unknown exception in Feature \"%s\" thrown\n",Feat->getNameInDocument()); - _RecomputeLog.push_back(new DocumentObjectExecReturn("Unknown exception!")); - Feat->setError(); - return true; + FC_ERR("Unknown exception in " << Feat->getFullName() << " thrown"); + d->addRecomputeLog("Unknown exception!",Feat); + return 1; } #endif - // error code - if (returnCode == DocumentObject::StdReturn) { + if(returnCode == DocumentObject::StdReturn) { Feat->resetError(); - } - else { + }else{ returnCode->Which = Feat; - _RecomputeLog.push_back(returnCode); + d->addRecomputeLog(returnCode); #ifdef FC_DEBUG - Base::Console().Error("Error in feature: %s\n%s\n",Feat->getNameInDocument(),returnCode->Why.c_str()); + FC_ERR("Failed to recompute " << Feat->getFullName() << ": " << returnCode->Why); +#else + FC_LOG("Failed to recompute " << Feat->getFullName() << ": " << returnCode->Why); #endif - Feat->setError(); + return 1; } - return false; + return 0; } -void Document::recomputeFeature(DocumentObject* Feat) +bool Document::recomputeFeature(DocumentObject* Feat, bool recursive) { - // delete recompute log - for (std::vector::iterator it=_RecomputeLog.begin();it!=_RecomputeLog.end();++it) - delete *it; - _RecomputeLog.clear(); + // delete recompute log + d->clearRecomputeLog(Feat); // verify that the feature is (active) part of the document if (Feat->getNameInDocument()) { - _recomputeFeature(Feat); - signalRecomputedObject(*Feat); - } + if(recursive) { + bool hasError = false; + recompute({Feat},true,&hasError); + return !hasError; + } else { + _recomputeFeature(Feat); + signalRecomputedObject(*Feat); + return Feat->isValid(); + } + }else + return false; } DocumentObject * Document::addObject(const char* sType, const char* pObjectName, @@ -2968,6 +3472,8 @@ DocumentObject * Document::addObject(const char* sType, const char* pObjectName, // mark the object as new (i.e. set status bit 2) and send the signal pcObject->setStatus(ObjectStatus::New, true); + pcObject->setStatus(ObjectStatus::PartialObject, isPartial); + if(!viewType) viewType = pcObject->getViewProviderNameOverride(); if(viewType) @@ -3177,8 +3683,36 @@ void Document::removeObject(const char* sName) if (pos == d->objectMap.end()) return; + if (pos->second->testStatus(ObjectStatus::PendingRecompute)) { + // TODO: shall we allow removal if there is active udno transaction? + FC_LOG("pending remove of " << sName << " after recomputing document " << getName()); + pos->second->setStatus(ObjectStatus::PendingRemove,true); + return; + } + _checkTransaction(pos->second,0,__LINE__); +#if 0 + if(!d->rollback && d->activeUndoTransaction && pos->second->hasChildElement()) { + // Preserve link group sub object global visibilities. Normally those + // claimed object should be hidden in global coordinate space. However, + // when the group is deleted, the user will naturally try to show the + // children, which may now in the global space. When the parent is + // undeleted, having its children shown in both the local and global + // coordinate space is very confusing. Hence, we preserve the visibility + // here + for(auto &sub : pos->second->getSubObjects()) { + if(sub.empty()) + continue; + if(sub[sub.size()-1]!='.') + sub += '.'; + auto sobj = pos->second->getSubObject(sub.c_str()); + if(sobj && sobj->getDocument()==this && !sobj->Visibility.getValue()) + d->activeUndoTransaction->addObjectChange(sobj,&sobj->Visibility); + } + } +#endif + if (d->activeObject == pos->second) d->activeObject = 0; @@ -3252,6 +3786,11 @@ void Document::removeObject(const char* sName) /// Remove an object out of the document (internal) void Document::_removeObject(DocumentObject* pcObject) { + if (testStatus(Document::Recomputing)) { + FC_ERR("Cannot delete " << pcObject->getFullName() << " while recomputing"); + return; + } + // TODO Refactoring: share code with Document::removeObject() (2015-09-01, Fat-Zer) _checkTransaction(pcObject,0,__LINE__); @@ -3322,165 +3861,191 @@ void Document::_removeObject(DocumentObject* pcObject) void Document::breakDependency(DocumentObject* pcObject, bool clear) { // Nullify all dependent objects - std::vector docObjs; - pcObject->ExpressionEngine.getDocumentObjectDeps(docObjs); - - for (std::map::iterator it = d->objectMap.begin(); it != d->objectMap.end(); ++it) { - std::map Map; - it->second->getPropertyMap(Map); - // search for all properties that could have a link to the object - for (std::map::iterator pt = Map.begin(); pt != Map.end(); ++pt) { - if (pt->second->getTypeId().isDerivedFrom(PropertyLink::getClassTypeId())) { - PropertyLink* link = static_cast(pt->second); - if (link->getValue() == pcObject) - link->setValue(0); - else if (link->getContainer() == pcObject && clear) - link->setValue(0); - } - else if (pt->second->getTypeId().isDerivedFrom(PropertyLinkSub::getClassTypeId())) { - PropertyLinkSub* link = static_cast(pt->second); - if (link->getValue() == pcObject) - link->setValue(0); - else if (link->getContainer() == pcObject && clear) - link->setValue(0); - } - else if (pt->second->getTypeId().isDerivedFrom(PropertyLinkList::getClassTypeId())) { - PropertyLinkList* link = static_cast(pt->second); - if (link->getContainer() == pcObject && clear) { - link->setValues(std::vector()); - } - else { - const auto &links = link->getValues(); - if (std::find(links.begin(), links.end(), pcObject) != links.end()) { - std::vector newLinks; - for(auto obj : links) { - if (obj != pcObject) - newLinks.push_back(obj); - } - link->setValues(newLinks); - } - } - } - else if (pt->second->getTypeId().isDerivedFrom(PropertyLinkSubList::getClassTypeId())) { - PropertyLinkSubList* link = static_cast(pt->second); - if (link->getContainer() == pcObject && clear) { - link->setValues(std::vector(), std::vector()); - } - else { - const std::vector& links = link->getValues(); - const std::vector& sub = link->getSubValues(); - std::vector newLinks; - std::vector newSub; - - if (std::find(links.begin(), links.end(), pcObject) != links.end()) { - std::vector::const_iterator jt; - std::vector::const_iterator kt; - for (jt = links.begin(),kt = sub.begin(); jt != links.end() && kt != sub.end(); ++jt, ++kt) { - if (*jt != pcObject) { - newLinks.push_back(*jt); - newSub.push_back(*kt); - } - } - - link->setValues(newLinks, newSub); - } - } - } - } - - if (std::find(docObjs.begin(), docObjs.end(), it->second) != docObjs.end()) { - std::vector paths; - pcObject->ExpressionEngine.getPathsToDocumentObject(it->second, paths); - for (std::vector::iterator jt = paths.begin(); jt != paths.end(); ++jt) { - // When nullifying the expression handle case where an identifier lacks of the property - try { - pcObject->ExpressionEngine.setValue(*jt, nullptr); - } - catch (const Base::Exception& e) { - e.ReportException(); - } - } - } - } + PropertyLinkBase::breakLinks(pcObject,d->objectArray,clear); } -DocumentObject* Document::copyObject(DocumentObject* obj, bool recursive) +std::vector Document::copyObject( + const std::vector &objs, bool recursive) { - std::vector objs; - objs.push_back(obj); + std::vector deps; + if(!recursive) + deps = objs; + else + deps = getDependencyList(objs,DepNoXLinked|DepSort); + if(!isSaved() && PropertyXLink::hasXLink(deps)) + throw Base::RuntimeError( + "Document must be saved at least once before link to external objects"); + MergeDocuments md(this); // if not copying recursively then suppress possible warnings md.setVerbose(recursive); - if (recursive) { - objs = obj->getDocument()->getDependencyList(objs); - auto it = std::find(objs.begin(), objs.end(), obj); - if (it != objs.end()) { - auto index = std::distance(objs.begin(), it); - std::swap(objs[index], objs.back()); + + unsigned int memsize=1000; // ~ for the meta-information + for (std::vector::iterator it = deps.begin(); it != deps.end(); ++it) + memsize += (*it)->getMemSize(); + + // if less than ~10 MB + bool use_buffer=(memsize < 0xA00000); + QByteArray res; + try { + res.reserve(memsize); + } + catch (const Base::MemoryException&) { + use_buffer = false; + } + + std::vector imported; + if (use_buffer) { + Base::ByteArrayOStreambuf obuf(res); + std::ostream ostr(&obuf); + exportObjects(deps, ostr); + + Base::ByteArrayIStreambuf ibuf(res); + std::istream istr(0); + istr.rdbuf(&ibuf); + imported = md.importObjects(istr); + } else { + static Base::FileInfo fi(App::Application::getTempFileName()); + Base::ofstream ostr(fi, std::ios::out | std::ios::binary); + exportObjects(deps, ostr); + ostr.close(); + + Base::ifstream istr(fi, std::ios::in | std::ios::binary); + imported = md.importObjects(istr); + } + + if(imported.size()!=deps.size()) + return imported; + + std::unordered_map indices; + size_t i=0; + for(auto o : deps) + indices[o] = i++; + std::vector result; + result.reserve(objs.size()); + for(auto o : objs) + result.push_back(imported[indices[o]]); + return result; +} + +std::vector +Document::importLinks(const std::vector &objArray) +{ + std::set links; + getLinksTo(links,0,GetLinkExternal,0,objArray); + + std::vector objs; + objs.insert(objs.end(),links.begin(),links.end()); + objs = App::Document::getDependencyList(objs); + if(objs.empty()) { + FC_ERR("nothing to import"); + return objs; + } + + for(auto it=objs.begin();it!=objs.end();) { + auto obj = *it; + if(obj->getDocument() == this) { + it = objs.erase(it); + continue; + } + ++it; + if(obj->testStatus(App::PartialObject)) { + throw Base::RuntimeError( + "Cannot import partial loaded object. Please reload the current document"); } } - unsigned int memsize=1000; // ~ for the meta-information - for (std::vector::iterator it = objs.begin(); it != objs.end(); ++it) - memsize += (*it)->getMemSize(); + Base::FileInfo fi(App::Application::getTempFileName()); + { + // save stuff to temp file + Base::ofstream str(fi, std::ios::out | std::ios::binary); + MergeDocuments mimeView(this); + exportObjects(objs, str); + str.close(); + } + Base::ifstream str(fi, std::ios::in | std::ios::binary); + MergeDocuments mimeView(this); + objs = mimeView.importObjects(str); + str.close(); + fi.deleteFile(); - QByteArray res; - res.reserve(memsize); - Base::ByteArrayOStreambuf obuf(res); - std::ostream ostr(&obuf); - this->exportObjects(objs, ostr); + const auto &nameMap = mimeView.getNameMap(); - Base::ByteArrayIStreambuf ibuf(res); - std::istream istr(0); - istr.rdbuf(&ibuf); - std::vector newObj = md.importObjects(istr); - if (newObj.empty()) - return 0; - else - return newObj.back(); + // First, find all link type properties that needs to be changed + std::map > propMap; + std::vector propList; + for(auto obj : links) { + propList.clear(); + obj->getPropertyList(propList); + for(auto prop : propList) { + auto linkProp = Base::freecad_dynamic_cast(prop); + if(linkProp && !prop->testStatus(Property::Immutable) && !obj->isReadOnly(prop)) { + auto copy = linkProp->CopyOnImportExternal(nameMap); + if(copy) + propMap[linkProp].reset(copy); + } + } + } + + // Then change them in one go. Note that we don't make change in previous + // loop, because a changed link property may break other depending link + // properties, e.g. a link sub refering to some sub object of an xlink, If + // that sub object is imported with a different name, and xlink is changed + // before this link sub, it will break. + for(auto &v : propMap) + v.first->Paste(*v.second); + + return objs; } DocumentObject* Document::moveObject(DocumentObject* obj, bool recursive) { + if(!obj) + return 0; Document* that = obj->getDocument(); if (that == this) return 0; // nothing todo - // all object of the other document that refer to this object must be nullified - that->breakDependency(obj, false); - std::string objname = getUniqueObjectName(obj->getNameInDocument()); - that->_removeObject(obj); - this->_addObject(obj, objname.c_str()); - obj->setDocument(this); - - std::map props; - obj->getPropertyMap(props); - for (std::map::iterator it = props.begin(); it != props.end(); ++it) { - if (it->second->getTypeId().isDerivedFrom(PropertyLink::getClassTypeId())) { - DocumentObject* link = static_cast(it->second)->getValue(); - if (recursive) { - moveObject(link, recursive); - static_cast(it->second)->setValue(link); - } - else { - static_cast(it->second)->setValue(0); - } - } - else if (it->second->getTypeId().isDerivedFrom(PropertyLinkList::getClassTypeId())) { - std::vector links = static_cast(it->second)->getValues(); - if (recursive) { - for (std::vector::iterator jt = links.begin(); jt != links.end(); ++jt) - moveObject(*jt, recursive); - static_cast(it->second)->setValues(links); - } - else { - static_cast(it->second)->setValues(std::vector()); - } - } + // True object move without copy is only safe when undo is off on both + // documents. + if(!recursive && !d->iUndoMode && !that->d->iUndoMode) { + // all object of the other document that refer to this object must be nullified + that->breakDependency(obj, false); + std::string objname = getUniqueObjectName(obj->getNameInDocument()); + that->_removeObject(obj); + this->_addObject(obj, objname.c_str()); + obj->setDocument(this); + return obj; } - return obj; + std::vector deps; + if(recursive) + deps = getDependencyList({obj},DepNoXLinked|DepSort); + else + deps.push_back(obj); + + auto objs = copyObject(deps,false); + if(objs.empty()) + return 0; + // Some object may delete its children if deleted, so we collect the IDs + // or all depdending objects for saftey reason. + std::vector ids; + ids.reserve(deps.size()); + for(auto o : deps) + ids.push_back(o->getID()); + + // We only remove object if it is the moving object or it has no + // depending objects, i.e. an empty inList, which is why we need to + // iterate the depending list backwards. + for(auto iter=ids.rbegin();iter!=ids.rend();++iter) { + auto o = that->getObjectByID(*iter); + if(!o) continue; + if(iter==ids.rbegin() + || o->getInList().empty()) + that->removeObject(o->getNameInDocument()); + } + return objs.back(); } DocumentObject * Document::getActiveObject(void) const @@ -3573,7 +4138,12 @@ std::string Document::getStandardObjectName(const char *Name, int d) const return Base::Tools::getUniqueName(Name, labels, d); } -std::vector Document::getObjects() const +std::vector Document::getDependingObjects() const +{ + return getDependencyList(d->objectArray); +} + +const std::vector &Document::getObjects() const { return d->objectArray; } @@ -3710,3 +4280,18 @@ Document::getPathsByOutList(const App::DocumentObject* from, const App::Document return array; } + +bool Document::mustExecute() const +{ + if(PropertyXLink::hasXLink(this)) { + bool touched = false; + _buildDependencyList(d->objectArray,false,0,0,0,&touched); + return touched; + } + + for (std::vector::const_iterator It = d->objectArray.begin();It != d->objectArray.end();++It) + if ((*It)->isTouched() || (*It)->mustExecute()==1) + return true; + return false; +} + diff --git a/src/App/Document.h b/src/App/Document.h index e7b256d7a8..da4d1804d7 100644 --- a/src/App/Document.h +++ b/src/App/Document.h @@ -70,6 +70,9 @@ public: Restoring = 3, Recomputing = 4, PartialRestore = 5, + Importing = 6, + PartialDoc = 7, + AllowPartialRecompute = 8, // allow recomputing editing object if SkipRecompute is set }; /** @name Properties */ @@ -157,11 +160,13 @@ public: Base::XMLReader&)> signalImportObjects; boost::signals2::signal&, Base::Reader&, const std::map&)> signalImportViewObjects; + boost::signals2::signal&)> signalFinishImportObjects; //signal starting a save action to a file boost::signals2::signal signalStartSave; //signal finishing a save action to a file boost::signals2::signal signalFinishSave; - boost::signals2::signal signalRecomputed; + boost::signals2::signal signalBeforeRecompute; + boost::signals2::signal&)> signalRecomputed; boost::signals2::signal signalRecomputedObject; //signal a new opened transaction boost::signals2::signal signalOpenTransaction; @@ -169,6 +174,8 @@ public: boost::signals2::signal signalCommitTransaction; // signal an aborted transaction boost::signals2::signal signalAbortTransaction; + boost::signals2::signal&)> signalSkipRecompute; + boost::signals2::signal signalFinishRestoreObject; boost::signals2::signal signalChangePropertyEditor; //@} @@ -181,10 +188,32 @@ public: bool saveAs(const char* file); bool saveCopy(const char* file) const; /// Restore the document from the file in Property Path - void restore (void); + void restore (const char *filename=0, + bool delaySignal=false, const std::set &objNames={}); + void afterRestore(bool checkPartial=false); + bool afterRestore(const std::vector &, bool checkPartial=false); + enum ExportStatus { + NotExporting, + Exporting, + }; + ExportStatus isExporting(const App::DocumentObject *obj) const; void exportObjects(const std::vector&, std::ostream&); void exportGraphviz(std::ostream&) const; std::vector importObjects(Base::XMLReader& reader); + /** Import any externally linked objects + * + * @param objs: input list of objects. Only objects belonging to this document will + * be checked for external links. And all found external linked object will be imported + * to this document. Link type properties of those input objects will be automatically + * reassigned to the imported objects. Note that the link properties of other objects + * in the document but not included in the input list, will not be affected even if they + * point to some object beining imported. To import all objects, simply pass in all objects + * of this document. + * + * @return the list of imported objects + */ + std::vector importLinks( + const std::vector &objs = {}); /// Opens the document from its file name //void open (void); /// Is the document already saved to a file? @@ -230,12 +259,15 @@ public: void addObject(DocumentObject*, const char* pObjectName=0); - /** Copy an object from another document to this document - * If \a recursive is true then all objects this object depends on - * are copied as well. By default \a recursive is false. - * Returns the copy of the object or 0 if the creation failed. + /** Copy objects from another document to this document + * + * @param recursive: if true, then all objects this object depends on are + * copied as well. By default \a recursive is false. + * + * @return Returns the list of objects copied. */ - DocumentObject* copyObject(DocumentObject* obj, bool recursive=false); + std::vector copyObject( + const std::vector &objs, bool recursive=false); /** Move an object from another document to this document * If \a recursive is true then all objects this object depends on * are moved as well. By default \a recursive is false. @@ -257,6 +289,8 @@ public: std::string getUniqueObjectName(const char *Name) const; /// Returns a name of the form prefix_number. d specifies the number of digits. std::string getStandardObjectName(const char *Name, int d) const; + /// Returns a list of document's objects including the dependencies + std::vector getDependingObjects() const; /// Returns a list of all Objects const std::vector &getObjects() const; std::vector getObjectsOfType(const Base::Type& typeId) const; @@ -277,18 +311,23 @@ public: void purgeTouched(); /// check if there is any touched object in this document bool isTouched(void) const; + /// check if there is any object must execute in this document + bool mustExecute(void) const; /// returns all touched objects std::vector getTouched(void) const; /// set the document to be closable, this is on by default. void setClosable(bool); /// check whether the document can be closed bool isClosable() const; - /// Recompute all touched features and return the number of recalculated features - int recompute(); + /** Recompute touched features and return the number of recalculated features + * + * @param objs: specify a sub set of objects to recompute. If empty, then + * all object in this document is checked for recompute + */ + int recompute(const std::vector &objs={}, + bool force=false,bool *hasError=0, int options=0); /// Recompute only one feature - void recomputeFeature(DocumentObject* Feat); - /// get the error log from the recompute run - const std::vector &getRecomputeLog(void)const{return _RecomputeLog;} + bool recomputeFeature(DocumentObject* Feat,bool recursive=false); /// get the text of the error of a specified object const char* getErrorDescription(const App::DocumentObject*) const; /// return the status bits @@ -379,11 +418,31 @@ public: bool checkOnCycle(void); /// get a list of all objects linking to the given object std::vector getInList(const DocumentObject* me) const; - /// Get a complete list of all objects the given objects depend on. The list - /// also contains the given objects! - /// deprecated! Use In- and OutList mimic in the DocumentObject instead! - std::vector getDependencyList - (const std::vector&) const; + + /// Option bit flags used by getDepenencyList() + enum DependencyOption { + /// Return topological sorted list + DepSort = 1, + /// Do no include object linked by PropertyXLink, as it can handle external link + DepNoXLinked = 2, + /// Raise exception on cycles + DepNoCycle = 4, + }; + /** Get a complete list of all objects the given objects depend on. + * + * This function is defined as static because it accpets objects from + * different documents, and the returned list will contain dependent + * objects from all relavent documents + * + * @param objs: input objects to query for dependency. + * @param options: See DependencyOption + */ + static std::vector getDependencyList( + const std::vector &objs, int options=0); + + std::vector getDependentDocuments(bool sort=true); + static std::vector getDependentDocuments(std::vector docs, bool sort); + // set Changed //void setChanged(DocumentObject* change); /// get a list of topological sorted objects (https://en.wikipedia.org/wiki/Topological_sorting) @@ -411,6 +470,11 @@ public: /// Check if there is any link to the given object bool hasLinksTo(const DocumentObject *obj) const; + /// Called by objects during restore to ask for recompute + void addRecomputeObject(DocumentObject *obj); + + const std::string &getOldLabel() const {return oldLabel;} + /// Function called to signal that an object identifier has been renamed void renameObjectIdentifiers(const std::map & paths, const std::function &selector = [](const App::DocumentObject *) { return true; }); @@ -418,6 +482,9 @@ public: virtual std::string getFullName() const override; + /// Indicate if there is any document restoring/importing + static bool isAnyRestoring(); + friend class Application; /// because of transaction handling friend class TransactionalObject; @@ -448,12 +515,14 @@ protected: /// callback from the Document objects after property was changed void onChangedProperty(const DocumentObject *Who, const Property *What); /// helper which Recompute only this feature - /// @return True if the recompute process of the Document shall be stopped, False if it shall be continued. - bool _recomputeFeature(DocumentObject* Feat); + /// @return 0 if succeed, 1 if failed, -1 if aborted by user. + int _recomputeFeature(DocumentObject* Feat); void _clearRedos(); /// refresh the internal dependency graph - void _rebuildDependencyList(void); + void _rebuildDependencyList( + const std::vector &objs = std::vector()); + std::string getTransientDirectoryName(const std::string& uuid, const std::string& filename) const; /** Open a new command Undo/Redo, an UTF-8 name can be specified diff --git a/src/App/DocumentObject.cpp b/src/App/DocumentObject.cpp index 4fb51929fd..b5e54cd652 100644 --- a/src/App/DocumentObject.cpp +++ b/src/App/DocumentObject.cpp @@ -123,11 +123,11 @@ DocumentObjectExecReturn *DocumentObject::execute(void) return StdReturn; } -bool DocumentObject::recomputeFeature() +bool DocumentObject::recomputeFeature(bool recursive) { Document* doc = this->getDocument(); if (doc) - doc->recomputeFeature(this); + return doc->recomputeFeature(this,recursive); return isValid(); } @@ -227,6 +227,26 @@ const char *DocumentObject::getNameInDocument() const return pcNameInDocument->c_str(); } +int DocumentObject::isExporting() const { + if(!getDocument() || !getNameInDocument()) + return 0; + return getDocument()->isExporting(this); +} + +std::string DocumentObject::getExportName(bool forced) const { + if(!pcNameInDocument) + return std::string(); + + if(!forced && !isExporting()) + return *pcNameInDocument; + + // '@' is an invalid character for an internal name, which ensures the + // following returned name will be unique in any document. Saving external + // object like that shall only happens in Document::exportObjects(). We + // shall strip out this '@' and the following document name during restoring. + return *pcNameInDocument + '@' + getDocument()->getName(); +} + bool DocumentObject::isAttachedToDocument() const { return (pcNameInDocument != 0); @@ -667,6 +687,19 @@ void DocumentObject::onChanged(const Property* prop) if(GetApplication().isClosingAll()) return; + if(!GetApplication().isRestoring() && + prop && !prop->testStatus(Property::PartialTrigger) && + getDocument() && + getDocument()->testStatus(Document::PartialDoc)) + { + static App::Document *warnedDoc; + if(warnedDoc != getDocument()) { + warnedDoc = getDocument(); + FC_WARN("Changes to partial loaded document will not be saved: " + << getFullName() << '.' << prop->getName()); + } + } + // Delay signaling view provider until the document object has handled the // change // if (_pDoc) @@ -753,7 +786,7 @@ DocumentObject *DocumentObject::getSubObject(const char *subname, // objects (think of the claimed children of a Fusion). But I do think we // should change that. if(transform && mat) { - auto pla = dynamic_cast(getPropertyByName("Placement")); + auto pla = Base::freecad_dynamic_cast(getPropertyByName("Placement")); if(pla) *mat *= pla->getValue().toMatrix(); } diff --git a/src/App/DocumentObject.h b/src/App/DocumentObject.h index ea2895a986..35d3439a3d 100644 --- a/src/App/DocumentObject.h +++ b/src/App/DocumentObject.h @@ -53,7 +53,14 @@ enum ObjectStatus { PythonCall = 6, Destroy = 7, Enforce = 8, - Expand = 16 + Recompute2 = 9, // set when the object is being recomputed in the second pass + PartialObject = 10, + PendingRecompute = 11, // set by Document, indicating the object is in recomputation queue + PendingRemove = 12, // set by Document, indicating the object is in pending for remove after recompute + ObjImporting = 13, // Mark the object as importing + NoTouch = 14, // no touch on any property change + GeoExcluded = 15, // mark as a member but not claimed by GeoFeatureGroup + Expand = 16, }; /** Return object for feature execution @@ -114,6 +121,8 @@ public: const char *getNameInDocument(void) const; /// Return the object ID that is unique within its owner document long getID() const {return _Id;} + /// returns the name that is safe to be exported to other document + std::string getExportName(bool forced=false) const; /// Return the object full name of the form DocName#ObjName virtual std::string getFullName() const override; virtual bool isAttachedToDocument() const; @@ -155,6 +164,8 @@ public: void setStatus(ObjectStatus pos, bool on) {StatusBits.set((size_t)pos, on);} //@} + int isExporting() const; + /** Child element handling */ //@{ @@ -275,8 +286,11 @@ public: */ virtual short mustExecute(void) const; - /// Recompute only this feature - bool recomputeFeature(); + /** Recompute only this feature + * + * @param recursive: set to true to recompute any dependent objects as well + */ + bool recomputeFeature(bool recursive=false); /// get the status Message const char *getStatusString(void) const; @@ -493,6 +507,14 @@ public: virtual bool adjustRelativeLinks(const std::set &inList, std::set *visited=0); + /** allow partial loading of dependent objects + * + * @return Returns 0 means do not support partial loading. 1 means allow + * dependent objects to be partially loaded, i.e. only create, but not + * restored. 2 means this object itself can be partially loaded. + */ + virtual int canLoadPartial() const {return 0;} + /** Allow object to redirect a subname path * * @param ss: input as the current subname path from \a topParent leading diff --git a/src/App/DocumentObjectPy.xml b/src/App/DocumentObjectPy.xml index e023fb8c67..02e22761df 100644 --- a/src/App/DocumentObjectPy.xml +++ b/src/App/DocumentObjectPy.xml @@ -57,7 +57,7 @@ - Recomputes this object + recompute(recursive=False): Recomputes this object @@ -250,12 +250,24 @@ or None if the GUI is not up + + + Check if the object must be recomputed + + + The unique identifier (among its document) of this object + + + Indicate if the object is being removed + + + A List of tuple(parent,subname) holding all parents to this object @@ -268,5 +280,11 @@ or None if the GUI is not up + + + Enable/disable no touch on any property change + + + diff --git a/src/App/DocumentObjectPyImp.cpp b/src/App/DocumentObjectPyImp.cpp index 50f3d65ae7..e21f918f9d 100644 --- a/src/App/DocumentObjectPyImp.cpp +++ b/src/App/DocumentObjectPyImp.cpp @@ -192,6 +192,9 @@ Py::List DocumentObjectPy::getState(void) const uptodate = false; list.append(Py::String("Recompute")); } + if (object->testStatus(App::Recompute2)) { + list.append(Py::String("Recompute2")); + } if (object->isRestoring()) { uptodate = false; list.append(Py::String("Restore")); @@ -199,6 +202,12 @@ Py::List DocumentObjectPy::getState(void) const if (object->testStatus(App::Expand)){ list.append(Py::String("Expanded")); } + if (object->testStatus(App::PartialObject)){ + list.append(Py::String("Partial")); + } + if (object->testStatus(App::ObjImporting)){ + list.append(Py::String("Importing")); + } if (uptodate) { list.append(Py::String("Up-to-date")); } @@ -361,11 +370,12 @@ PyObject* DocumentObjectPy::setExpression(PyObject * args) PyObject* DocumentObjectPy::recompute(PyObject *args) { - if (!PyArg_ParseTuple(args, "")) + PyObject *recursive=Py_False; + if (!PyArg_ParseTuple(args, "|O",&recursive)) return NULL; try { - bool ok = getDocumentObjectPtr()->recomputeFeature(); + bool ok = getDocumentObjectPtr()->recomputeFeature(PyObject_IsTrue(recursive)); return Py_BuildValue("O", (ok ? Py_True : Py_False)); } catch (const Base::Exception& e) { @@ -650,6 +660,16 @@ PyObject* DocumentObjectPy::getParentGeoFeatureGroup(PyObject *args) } } +Py::Boolean DocumentObjectPy::getMustExecute() const +{ + try { + return Py::Boolean(getDocumentObjectPtr()->mustExecute()?true:false); + } + catch (const Base::Exception& e) { + throw Py::RuntimeError(e.what()); + } +} + PyObject* DocumentObjectPy::getPathsByOutList(PyObject *args) { PyObject* o; @@ -838,3 +858,11 @@ PyObject *DocumentObjectPy::adjustRelativeLinks(PyObject *args) { Py::String DocumentObjectPy::getOldLabel() const { return Py::String(getDocumentObjectPtr()->getOldLabel()); } + +Py::Boolean DocumentObjectPy::getNoTouch() const { + return Py::Boolean(getDocumentObjectPtr()->testStatus(ObjectStatus::NoTouch)); +} + +void DocumentObjectPy::setNoTouch(Py::Boolean value) { + getDocumentObjectPtr()->setStatus(ObjectStatus::NoTouch,value.isTrue()); +} diff --git a/src/App/DocumentPy.xml b/src/App/DocumentPy.xml index 055740b4e8..d49cbdf345 100644 --- a/src/App/DocumentPy.xml +++ b/src/App/DocumentPy.xml @@ -93,14 +93,38 @@ viewType (String): override the view provider type directly, only effective when - copyObject(object, bool with_dependencies = False, bool ignored_argument = False) - Copy an object from another document to this document. If with_dependencies is True, all objects this object depends on are copied too. + +copyObject(object, with_dependencies=False) +Copy an object or objects from another document to this document. + +object: can either a single object or sequence of objects +with_dependencies: if True, all internal dependent objects are copied too. + - moveObject(object, bool with_dependencies = False) - Transfers an object from another document to this document. If with_dependencies is True, all objects this object depends on are transferred too. + +moveObject(object, bool with_dependencies = False) +Transfers an object from another document to this document. + +object: can either a single object or sequence of objects +with_dependencies: if True, all internal dependent objects are copied too. + + + + + + +importLinks(object|[object...]) + +Import any externally linked object given a list of objects in +this document. Any link type properties of the input objects +will be automatically reassigned to the imported object + +If no object is given as input, it import all externally linked +object of this document. + @@ -120,7 +144,7 @@ viewType (String): override the view provider type directly, only effective when - Recompute the document and returns the amount of recomputed features + recompute(objs=None): Recompute the document and returns the amount of recomputed features @@ -144,10 +168,10 @@ Both parameters are optional. - getLinksTo(obj, options=0, maxCount=0): return objects linked to 'obj' +getLinksTo(obj, options=0, maxCount=0): return objects linked to 'obj' - options: 1: recursive, 2: check link array. Options can combine. - maxCount: to limit the number of links returned +options: 1: recursive, 2: check link array. Options can combine. +maxCount: to limit the number of links returned @@ -161,6 +185,17 @@ Both parameters are optional. Returns a file name with path in the temp directory of the document. + + + +getDependentDocuments(sort=True) + +Returns a list of documents that this document directly or indirectly links to including itself. + +sort: whether to topologically sort the return list + + + The dependency graph as GraphViz text @@ -245,12 +280,54 @@ Both parameters are optional. + + + A list of all documents that link to this document. + + + + + + A list of all documents that this document links to. + + + + + + Indicate if the document is restoring + + + + + + Indicate if the document is partially loaded + + + + + + Indicate if the document is importing. Note the document will also report Restoring while importing + + + + + + Indicate if the document is recomputing + + + Indicate whether the document is undoing/redoing + + + Contains the old label before change + + + diff --git a/src/App/DocumentPyImp.cpp b/src/App/DocumentPyImp.cpp index 255d10188e..d8f77c39e4 100644 --- a/src/App/DocumentPyImp.cpp +++ b/src/App/DocumentPyImp.cpp @@ -289,21 +289,77 @@ PyObject* DocumentPy::removeObject(PyObject *args) PyObject* DocumentPy::copyObject(PyObject *args) { - // 'keep' is not needed any more but leave it there for backward compatibility - PyObject *obj, *rec=Py_False, *keep=Py_False; - if (!PyArg_ParseTuple(args, "O!|O!O!",&(DocumentObjectPy::Type),&obj,&PyBool_Type,&rec,&PyBool_Type,&keep)) + PyObject *obj, *rec=Py_False; + if (!PyArg_ParseTuple(args, "O|O",&obj,&rec)) return NULL; // NULL triggers exception - DocumentObjectPy* docObj = static_cast(obj); - DocumentObject* copy = getDocumentPtr()->copyObject(docObj->getDocumentObjectPtr(), - PyObject_IsTrue(rec) ? true : false); - if (copy) { - return copy->getPyObject(); - } - else { - std::string str("Failed to copy the object"); - throw Py::Exception(Base::BaseExceptionFreeCADError,str); + std::vector objs; + bool single = false; + if(PySequence_Check(obj)) { + Py::Sequence seq(obj); + for(size_t i=0;i(seq[i].ptr())->getDocumentObjectPtr()); + } + }else if(!PyObject_TypeCheck(obj,&DocumentObjectPy::Type)) { + PyErr_SetString(PyExc_TypeError, + "Expect first argument to be either a document object or sequence of document objects"); + return 0; + }else { + objs.push_back(static_cast(obj)->getDocumentObjectPtr()); + single = true; } + + PY_TRY { + auto ret = getDocumentPtr()->copyObject(objs,PyObject_IsTrue(rec)); + if(ret.size()==1 && single) + return ret[0]->getPyObject(); + + Py::Tuple tuple(ret.size()); + for(size_t i=0;igetPyObject(),true)); + return Py::new_reference_to(tuple); + }PY_CATCH +} + +PyObject* DocumentPy::importLinks(PyObject *args) +{ + PyObject *obj = Py_None; + if (!PyArg_ParseTuple(args, "|O",&obj)) + return NULL; // NULL triggers exception + + std::vector objs; + if(PySequence_Check(obj)) { + Py::Sequence seq(obj); + for(size_t i=0;i(seq[i].ptr())->getDocumentObjectPtr()); + } + }else if(obj == Py_None) { + }else if(!PyObject_TypeCheck(obj,&DocumentObjectPy::Type)) { + PyErr_SetString(PyExc_TypeError, + "Expect first argument to be either a document object or sequence of document objects"); + return 0; + }else + objs.push_back(static_cast(obj)->getDocumentObjectPtr()); + + if(objs.empty()) + objs = getDocumentPtr()->getObjects(); + + PY_TRY { + auto ret = getDocumentPtr()->importLinks(objs); + + Py::Tuple tuple(ret.size()); + for(size_t i=0;igetPyObject(),true)); + return Py::new_reference_to(tuple); + }PY_CATCH } PyObject* DocumentPy::moveObject(PyObject *args) @@ -405,16 +461,34 @@ PyObject* DocumentPy::clearUndos(PyObject * args) PyObject* DocumentPy::recompute(PyObject * args) { - if (!PyArg_ParseTuple(args, "")) // convert args: Python->C + PyObject *pyobjs = Py_None; + PyObject *force = Py_False; + PyObject *checkCycle = Py_False; + if (!PyArg_ParseTuple(args, "|OO!O!",&pyobjs, + &PyBool_Type,&force,&PyBool_Type,&checkCycle)) // convert args: Python->C return NULL; // NULL triggers exception - try { - int objectCount = getDocumentPtr()->recompute(); + PY_TRY { + std::vector objs; + if(pyobjs!=Py_None) { + if(!PySequence_Check(pyobjs)) { + PyErr_SetString(PyExc_TypeError, "expect input of sequence of document objects"); + return 0; + } + Py::Sequence seq(pyobjs); + for(size_t i=0;i(seq[i].ptr())->getDocumentObjectPtr()); + } + } + int options = 0; + if(PyObject_IsTrue(checkCycle)) + options = Document::DepNoCycle; + int objectCount = getDocumentPtr()->recompute(objs,PyObject_IsTrue(force),0,options); return Py::new_reference_to(Py::Int(objectCount)); - } - catch (const Base::RuntimeError& e) { - PyErr_SetString(PyExc_RuntimeError, e.what()); - return 0; - } + } PY_CATCH; } PyObject* DocumentPy::getObject(PyObject *args) @@ -727,9 +801,68 @@ PyObject* DocumentPy::getLinksTo(PyObject *args) ret.setItem(i++,Py::Object(o->getPyObject(),true)); return Py::new_reference_to(ret); }PY_CATCH +} + +Py::List DocumentPy::getInList(void) const +{ + Py::List ret; + auto lists = PropertyXLink::getDocumentInList(getDocumentPtr()); + if(lists.size()==1) { + for(auto doc : lists.begin()->second) + ret.append(Py::Object(doc->getPyObject(), true)); + } + return ret; +} + +Py::List DocumentPy::getOutList(void) const +{ + Py::List ret; + auto lists = PropertyXLink::getDocumentOutList(getDocumentPtr()); + if(lists.size()==1) { + for(auto doc : lists.begin()->second) + ret.append(Py::Object(doc->getPyObject(), true)); + } + return ret; +} + +PyObject *DocumentPy::getDependentDocuments(PyObject *args) { + PyObject *sort = Py_True; + if (!PyArg_ParseTuple(args, "|O", &sort)) + return 0; + PY_TRY { + auto docs = getDocumentPtr()->getDependentDocuments(PyObject_IsTrue(sort)); + Py::List ret; + for(auto doc : docs) + ret.append(Py::Object(doc->getPyObject(), true)); + return Py::new_reference_to(ret); + } PY_CATCH; +} + +Py::Boolean DocumentPy::getRestoring(void) const +{ + return Py::Boolean(getDocumentPtr()->testStatus(Document::Status::Restoring)); +} + +Py::Boolean DocumentPy::getPartial(void) const +{ + return Py::Boolean(getDocumentPtr()->testStatus(Document::Status::PartialDoc)); +} + +Py::Boolean DocumentPy::getImporting(void) const +{ + return Py::Boolean(getDocumentPtr()->testStatus(Document::Status::Importing)); +} + +Py::Boolean DocumentPy::getRecomputing(void) const +{ + return Py::Boolean(getDocumentPtr()->testStatus(Document::Status::Recomputing)); +} + Py::Boolean DocumentPy::getTransacting() const { return Py::Boolean(getDocumentPtr()->isPerformingTransaction()); } +Py::String DocumentPy::getOldLabel() const { + return Py::String(getDocumentPtr()->getOldLabel()); } diff --git a/src/App/MergeDocuments.cpp b/src/App/MergeDocuments.cpp index 819a7c364e..101c46d2b2 100644 --- a/src/App/MergeDocuments.cpp +++ b/src/App/MergeDocuments.cpp @@ -63,6 +63,14 @@ public: return true; } protected: + + // It is not safe to change potential object name reference at this level. + // For example, a LinkSub with sub element name Face1 may also be some + // object's name that may potentially be mapped. In addition, with the + // introduction of full quanlified SubName reference, the Sub value inside + // LinkSub may require customized mapping. So we move the mapping logic to + // various link property's Restore() function. +#if 0 void startElement(const XMLCh* const uri, const XMLCh* const localname, const XMLCh* const qname, const XERCES_CPP_NAMESPACE_QUALIFIER Attributes& attrs) @@ -107,6 +115,7 @@ protected: if (LocalName == "Property") propertyStack.pop(); } +#endif private: std::map& nameMap; diff --git a/src/App/MergeDocuments.h b/src/App/MergeDocuments.h index 4e2249240e..dc89f1db16 100644 --- a/src/App/MergeDocuments.h +++ b/src/App/MergeDocuments.h @@ -50,6 +50,8 @@ public: void SaveDocFile (Base::Writer & w) const; void RestoreDocFile(Base::Reader & r); + const std::map &getNameMap() const {return nameMap;} + private: bool guiup; bool verbose; diff --git a/src/App/Property.h b/src/App/Property.h index 666fb8f0b0..7db2d21a65 100644 --- a/src/App/Property.h +++ b/src/App/Property.h @@ -155,10 +155,35 @@ public: /// Get valid paths for this property; used by auto completer virtual void getPaths(std::vector & paths) const; - /// Called at the begining of Document::afterRestore(). See comments there. + /** Called at the begining of Document::afterRestore() + * + * This function is called without dependency sorting, because some + * types of link property can only reconstructs the linking information + * inside this function. + * + * One example use case of this function is PropertyLinkSub that uses + * afterRestore() to parse and restore subname references, which may + * contain sub-object reference from external document, and there will be + * special mapping required during object import. + * + * Another example is PropertyExpressionEngine which only parse the + * restored expression in afterRestore(). The reason, in addition to + * subname mapping like PropertyLinkSub, is that it can handle document + * name adjustment as well. It internally relies on PropertyXLink to store + * the external document path for external linking. When the extenal + * document is restored, its internal name may change due to name conflict + * with existing documents. PropertyExpressionEngine can now auto adjust + * external references without any problem. + */ virtual void afterRestore() {} - /// Called before calling DocumentObject::onDocumentRestored() + /** Called before calling DocumentObject::onDocumentRestored() + * + * This function is called after finished calling Property::afterRestore() + * of all properies of objects. By then, the object dependency information + * is assumed ready. So, unlike Property::afterRestore(), this function is + * called on objects with dependency order. + */ virtual void onContainerRestored() {} /** Property status handling diff --git a/src/Base/Sequencer.h b/src/Base/Sequencer.h index bfef5cb825..e5aab10666 100644 --- a/src/Base/Sequencer.h +++ b/src/Base/Sequencer.h @@ -150,6 +150,9 @@ public: */ bool wasCanceled() const; + /// Check if the operation is aborted by user + virtual void checkAbort() {} + protected: /** * Starts a new operation, returns false if there is already a pending operation, diff --git a/src/Gui/MergeDocuments.cpp b/src/Gui/MergeDocuments.cpp index 93316f4fa5..a32687b604 100644 --- a/src/Gui/MergeDocuments.cpp +++ b/src/Gui/MergeDocuments.cpp @@ -63,6 +63,9 @@ public: return true; } protected: + // See App::MergeDocument::XMLMergeReader for comments, with one additional + // benefits, we can save repetitive coding here. +#if 0 void startElement(const XMLCh* const uri, const XMLCh* const localname, const XMLCh* const qname, const XERCES_CPP_NAMESPACE_QUALIFIER Attributes& attrs) @@ -107,6 +110,7 @@ protected: if (LocalName == "Property") propertyStack.pop(); } +#endif private: std::map& nameMap;