Oracle Endeca Information Discovery Studio and Integrator diagnostic Files

The purpose of this document is to provide information on useful diagnostic logs and data to collect for the Integrator component that is included in the following Endeca products:

Oracle Endeca Information Discovery Studio
Oracle Endeca Information Discovery Integrator

Integrator Designer

Integrator project directory (containing graphs, data-in, config-in, data-out folders, etc..) from Designer workspace would provide information about the projects

Console's output would also provide information around failures once job is fired. There is limited buffer in the console output window in the Integrator Designer. If the console output window does not show the full output, right click on the graph > Run As > Run Configurations > Common > Standard Input and Output > File: and specify an actual log file to record the console output information, then run the graph again to collect the full log and send to us.

Integrator Server

Application server log files can be found at  (<tomcat_dir>/logs or <weblogic_domain_dir>/servers/<server_name>/logs).

Integrator graph run logs are generally at <tomcat_dir>/temp/cloverlogs or <system_tmp_dir>/cloverlogs (if running on Weblogic).

Sandbox directory (this is specified when you first defined your sandbox for the Integrator Server. Check out the "Sandbox root path" from the Integrator Server admin console > Sandboxes to identify the location of your graph's sandbox directory)

Debugging Bulk Ingest Issues

When troubleshooting bulk load issues (when using Bulk Add/Replace component), it is often useful to include --bulkingestverbose and --datalayerverbose in the Dgraph startup command/arguments.

To add the above parameters to Dgraph/Data Store/Data Domain startup command for 3.X version

Create a data domain profile (http://docs.oracle.com/cd/E37502_01/server.751/es_admin/toc.htm#Data%20domain%20profile%20operations ) with the appropriate logging arguments. Export (export-dd) the data domain to back it up somewhere. Disable and delete the data domain. And use import-dd to import the backed up domain while referencing the new data domain profile. This will effectively "recreate" the data domain with additional Dgraph startup arguments. Please refer t http://docs.oracle.com/cd/E37502_01/server.751/es_admin/toc.htm#Data%20domain%20operaions for details on data domain operations/commands

Add the 2 commands/arguments noted above to your Dgraph/Data Store/Data Domain, reproduce the bulk ingest problem and then provide the latest Dgraph/Data Store/Data Domain standard out/error log (refer to 1557615.1 for the location of the standard out/err log).

Oracle Support Document

[Diagnostic Logs and Data Gathering] Oracle Endeca Information Discovery > Integrator (Doc ID 1558013.1)

I hope you find this information useful.

ODI 11g/12 C - How to Load Data into a Flat File

Oracle Data Integrator supports heterogeneous sources and targets. It is a common requirement while working with ODI to load data into a flat file (.txt or .csv). This blog is an attempt to list down the specific steps to carry out data load into flat files along with information on which data modules should we use for such an operation.

ODI Restrictions

For loading data into files, ODI has following two restrictions.

Flat files can be used a Source or Target but not as Staging Area. When loading data into a file, the Staging Area must to choose different than that of target. In short the target checkbox on the definition tab of integration interface should be different than target. The staging area should be a schema than a file.

Agent should have direct access to the file system ( local or mounted network disk partition) on which the file resides.

ODI Knowledge Modules

Knowledge Modules (KMs) are code templates. Each KM is dedicated to an individual task in the overall data integration process. The code in the KMs appears in nearly the form that it will be executed except that it includes Oracle Data Integrator (ODI) substitution methods enabling it to be used generically by many different integration jobs. The code that is generated and executed is derived from the declarative rules and metadata defined in the ODI Designer module.

The features of KMs are

#1 – A KM will be reused across several interfaces or models. Change once used across all transformations is a mantra.

# 2 – KMs can be analyzed for impact analysis

# 3 – KMs execution require metadata from interfaces, data stores and models

ODI KM Categories

KM fall into 6 different categories

Reverse-engineering KM – Retrieves metadata to the ODI work repository, it is been used in models to perform a customized reverse-engineering

Check KM - Checks consistency of data against constraints. Theses KMs are used in models and sub models and data store for data integrity audit. It is also used in interfaces for flow control or static control.

Loading KM - Loads heterogeneous data to a staging area. It is used in interfaces with heterogeneous sources

Integration KM – Integrates data from the staging area to a target used in interfaces

Journalizing KM – Creates the Change Data Capture framework objects in the source staging area. This KM is used in models, sub models and data stores to create, start and stop journals and to register subscribers.

Service KM - Generates data manipulation web services. It is used in models and data stores

KM Required to Data Load to Flat Files

There are potentially we need one or two KMs.

The choice of Loading KM (LKM) will depend on the choice of staging area. If staging area is same as the source schema then no LKM is required.  If is different than the source schema, an LKM would be in charge of loading source data from a remote server to the staging area. The LKM implements the declarative rules that need to be executed on the source server and retrieves a single result set that it stores in a "C$" table in the staging area.

The Integration KM (IKM) is in charge of writing the final, transformed data to the target table/File. IKM simply execute the Staging and Target transformations, joins and filters on staging tables (c$ and other tables). For staging area to file IKM SQL to File Append IKM is the best suitable IKM.

Oracle Support Document

How To Load Data Into a Flat File Using ODI (Doc ID 976092.1)

I hope you find this information useful.

Oracle Endeca Information Discovery Resources

Oracle Endeca Information Discovery (OEID) is one the most significant product to enter into the Big Data space, this blog is an attempt to give a brief overview of the OEID and list down links for resources for the same.

Overview

Oracle Endeca Information Discovery is a data discovery platform that guides organizations to better decisions based on diverse and changing data. It leverages the Endeca Server, a hybrid search-analytical database that provides a centralized platform to rapidly deploy interactive analytic applications and keep pace with changing business requirements while maintaining information governance. 

Oracle Endeca Information Discovery consists of the following modules:

Studio includes the Provisioning Service, which allows users to upload data directly from spreadsheets or JSON files, or import data from configured Oracle BI Server or JDBC connections.

Search, Guided Navigation, and analytics capabilities powered by Oracle Endeca Server.

Data integration options which includes Integrator ETL and Integrator Acquisition System. - Integrator ETL provides connectors for standard structured and unstructured data sources, a data enrichment library, and direct connectors to the Oracle Endeca Server.

Oracle Assets for OEID

Software Download

Download the software from the Software Delivery Cloud https://edelivery.oracle.com
- Select a Product Pack of Oracle Endeca and the appropriate Platform.
- Click Go
- Then choose the appropriate Media Pack.
Documentation
Oracle Commerce Guided Search (Endeca Guided Search)

http://www.oracle.com/technetwork/indexes/documentation/endecaguidedsearch-1552767.html
Oracle Endeca Information Discovery
http://www.oracle.com/technetwork/middleware/endeca/documentation/index.html

Knowledge, Technical WIKI, Solution Articles
https://support.oracle.com/
- Select Knowledge tab
- In the Browse Knowledge section search for 'Endeca'
- Select Oracle Endeca Information Discovery Studio or Oracle Commerce Guided Search / Endeca Experience Manager
- Click Search
My Oracle Support Community
Endeca (Oracle Commerce Guided Search)
https://community.oracle.com/community/support/oracle_commerce/endeca

Oracle Endeca Information Discovery
https://community.oracle.com/community/support/business_intelligence/oracle_endeca_information_discovery

Oracle Support Document

Where to Find Oracle Endeca Resources (Doc ID 1444696.1)

I hope you find this information useful.

OBIEE 11g –Not Designed to do it or Can’t do List

I believe OBIEE is one of the best business intelligence tools in market now a day. There are number of things which OBIEE 11g can do and lots of stuff has been written about it. Are there any things which OBIEE 11G can’t do? It is tricky question and will not found this in any standard Oracle documentation which is quite oblivious. During number of implementations I came across few of the limitations (rather OBIEE is not designed to do it).

This blog is an attempt to list down the stuffs which OBIEE 11g can’t do or not designed to do.

Item A: 

OBIEE 11g does not support multiple prompts created on the same column. For e.g. Create Year > 2000 and Create Year < 2015 does not work.

Workaround A

I would suggest using a proper operator rather than using one column for two or more times. E.g. Create Year Between 2000 and 2015 would work

Oracle Support Document

OBIEE 11g Multiple Prompts Created on the Same Column does not Work (Doc ID 1925214.1)

Item B: 

OBIEE 11g does not support sort order of the attribute columns when these columns are being used with the hierarchy columns.

Workaround B

Developer should not mix hierarchical columns with the attribute columns that make up the hierarchy.

Oracle Support Document

OBIEE 11g Sort Order not Working when Using Hierarchy Column (Doc ID 1924905.1)

Item C: 

OBIEE 11g Administrator Import Metadata does not support static variables in the connection pool due to security reasons

Workaround C

This can be achieved by using session variable with default value , check following blog entry http://bidirect.blogspot.co.uk/2014/09/obiee-11g-use-of-dynamic-and-static.html

Oracle Support Document

OBIEE 11g about the Use of Static Variables in the Connection Pool (Doc ID 1925713.1)

Inference 

This been removed for security reason. I hope you find this information useful.

Item D: 

OBIEE 11g - when editing a filter one can’t select a value bigger than 500

Workaround D

Apply the one-off Patch:17413638 SPINBOX IN EDIT FILTER CAN'T SELECT VALUE BIGGER THAN 500

Oracle Support Document

OBIEE 11g: When Editing a Filter You Can't Select a Value Bigger Than 500 (Doc ID 1622271.1)

Item E: 

OBIEE 11g – When database objects set to NUMBER (xx,y) are imported into OBIEE repository, all the number data types are shown as an INT data type. So all large numbers to report as 2147483647

Workaround E

After importing set the database objects as DOUBLE. It is shame as OBIEE import does not automatically convert Number( XX.Y) data type into Double.

Oracle Support Document

OBIEE 11g Is There an Alternative Datatype As NUMBER Is Not Large Enough In Repository.

(Doc ID 1625091.1)

Item F: 

OBIEE 11g – When using OBIEE 11g with Google Maps Map viewer, two Zoom Controls are displayed. One in OBIEE and one in Google itself.

Workaround F

No workaround available. This is caused by a recent change by Google in their mapping API.

Oracle Support Document

Using OBIEE 11g with Google Maps In Mapviewer, Two Zoom Controls Are Displayed (Doc ID 1639735.1)

Item G: 

OBIEE 11g – There is no quick way or API based solution to change all subject area permission from Authenticated User to BI Administrator in OBIEE via Analytics -> Administration- Manage Privilege webpage

Workaround G

No workaround available. This needs to be done manually L

Oracle Support Document

OBIEE 11g FAQ: Is There Any Method to Change Presentation Services Subject Area Permissions Programmatically or With a Batch Process (Doc ID 1681133.1)

Item H: 

OBIEE 11g – After moving one catalog through archive/unarchive  or copy/paste, the objects in the catalog looks fine, however the items referenced by the dashboards are invalid because the paths were not updated during the migration process

Workaround H

A potential solution may be to try use catalog manager's search and replace functionality to replace the references to the old path with the new location

Oracle Support Document

OBIEE 11g Dashboards Migration Issue (Doc ID 1670281.1)

Inference 

I believe no product is complete or possible to design to cover all features from the wish list, so OBIEE is not an exception. The idea is to make sure that if does not work normally what is the best workaround to start with. I believe Oracle eventually will work on some reasonable requests as enhancements in future releases of OBIEE 11g.