OBIEE 11.1.1.7.1 – Size of Repository – Is there any Restriction on Size? When it is too large to load via Enterprise Manager?

Is there a size limitation or any other limitations for the OBIEE 11g RPD?  Is it depends on operating system allowed file size or is there any limitation from Middleware?  How the sizes of the repository affect online access or start and stop processes?  Is it different in 32 bit than 64 bit? Does OBIEE 11g allow bigger size repository than OBIEE 10g?

As mentioned there are numbers of questions regarding maximum size for OBIEE 11g repository. This blog is an attempt to answer all these questions and share answers to larger audience.

Does OBIEE 11g Support Multiple Repositories? Can we use multiple repositories with smaller sizes?

OBIEE 11g does not support more than one repository per installation. The intent is that you cannot have multiple repositories deployed in a domain. This does not mean you cannot use a vertical or horizontal scale to have more than one BI Server loading the repository. So we have to use one repository per installation, hence there is no option to use multiple repositories with smaller sizes, it is shame.

Impact of Size on Online Mode Access

Opening repository in on-line mode loads repository metadata in RAM.  The slow opening of repository is a reflection not just the size of the RPD, but the power of the server BI is running on and the network connection between the client and the server. In short size of repository is a factor but not only factor.

Can Repository Size – X TiB or YPiB?

I haven’t come across any documentation which talks about the actual limit on the size of RPD for OBIEE 10g and/or OBIEE 11g. It is very normal that it can get bigger and bigger in case of multiple BI Apps implementations. I am assuming that one can design repository with XTiB or YPiB, there is no restriction. Good luck for those we would like to do so.

Is Operating System File Size Limitation Affect RPD Size?

The limit of size of file depends on the flavor of UNIX and also the file system and addressable memory size. The general perception about UNIX is that maximum size on UNIX supported is 2.0 GB, which is based on old hardware and kernel version.

The latest limitations for Solaris UNIX are as follows

UFS1: 4GiB to 256TiB
UFS2 : 512GiB to 32PiB
ZFS: 16Exbyte Max file size

In short it seems that there is no restriction on file size for repository based on which operating system on OBIEE 11g is running.

How Much Enterprise Manager Can Handle for Upload RPD ?  

For repository file is bigger than 40MB, Enterprise Manager throws an error while repository upload process, as 40MB is the maximum size that Enterprise Manager can handle.  This is an Enterprise Manager limit and is specified in web.xml file. So this limit is not a limit to Size of repository, it is a limitation to middleware.

One option to overcome this limit is to check if it is feasible to increase the maximum file size specified in web.xml. There is a work-around, which is to manually deploy the repository. OBIEE 11g RPD can be deployed by using MBean via Enterprise Manager and A WLST Script.  An extract from Oracle Support document which talks about RPD upload via Enterprise Manager and WLST Script are as follows.

Please refer tech- notes listed below for more information

1. How to Deploy an OBIEE 11g RPD Using An MBean Via Enterprise Manager And A WLST Script (Doc ID 1319333.1)

2. Using WLST to Manage the Deployment of Repository/Catalog at Business Intelligence Server (Doc ID 1319303.1)

How to Deploy RPD using MBean in Enterprise Manager

Steps to deploy RPD using MBean in Enterprise Manager are as follows, this is used for Repository size is bigger than 40MB, assuming that it is limit specified within web.xml

• Navigate to and login to EM Fusion Middleware Control URL (http://host.domain:7001/em)
• Expand the Web Logic Domain from the menu located at the left. Continue expanding until finding the Managed Server, usually called bi_server1 by default
• In the right pane, you will see a new page. From the Web Logic Server menu at the top, choose System M Bean Browser
• Expand 'Application Defined MBeans' > oracle.biee.admin > Domain: bifoundation_domain > BIDomain > and select the first BIDomain , it could be the second, depending upon your version, but it will have an 'Operations' tab
• Navigate to the Operations Tab > Select the ’lock’ link > Press the 'Invoke' button.
• Navigate to BIDomain.BIInstance.ServerConfiguration (which is in the same path) and select BIDomain.BIInstance.ServerConfiguration Navigate to the 'Operations' Tab Choose " uploadRepository
• Enter a path for the RPD and its password. For example: C:\OBI11\instances\instance3\bifoundation\OracleBIServerComponent\coreapplication_obis1\repository\repositoryname.rpd
• Press "Invoke"
• Release the lock on the domain. Navigate back to BIDomain > and select the first BIDomain Navigate to the 'Operations' tab and select the first "Commit" option and then press "Invoke"
• Re-start the BI Server (mandatory)

How to Deploy RPD Using WLST

This section includes deployment of the repository/catalog with WLST and script to change the default repository file with Jython script. To carry out these steps Web Logic Admin Server should be up and running, new repository should located in machine where BI server is running and user should have administrative rights

• Save the content bellow as updateRepository.py

• Script
   
  username = "weblogic"
  password = "weblogic1"
  newRPDlocation = "<local-path-to-RPD>"
  newRPDpassword = "<new-RPD-password>"
  connect(username, password, "t3://host:port");
  print 'Connecting to Domain ...'
  domainCustom()
  cd ('oracle.biee.admin')
  print 'Connecting to BIDomain MBean ...'
  cd ('oracle.biee.admin:type=BIDomain,group=Service')
  objs = jarray.array([], java.lang.Object)
  strs = jarray.array([], java.lang.String)
  invoke('lock', objs, strs)
  cd ('..')
  cd ('oracle.biee.admin:type=BIDomain.BIInstance.ServerConfiguration,biInstance=coreapplication,group=Service')
  objs=jarray.array([newRPDlocation,newRPDpassword],Object)
  strs=jarray.array(['java.lang.String', 'java.lang.String'],String)
  invoke('uploadRepository', objs, strs)
  cd ('..')
  cd ('oracle.biee.admin:type=BIDomain,group=Service')
  objs = jarray.array([], java.lang.Object)
  strs = jarray.array([], java.lang.String)
  invoke('commit', objs, strs)
  disconnect()
  exit()

• Modify the parameters for the user, connection string to the Admin Server and new RPD information.
• Set the domain environment using the script. $ WL_HOME/user_projects/domains/<DOMAIN>/. bin/setDomainEnv.sh 
• Run the script as follow: $ java weblogic.WLST updateRepository.py,

• Sample Script Output
  
  >>:~/WLS/10.3.3.0/user_projects/domains/3-2825937801/repo$ java weblogic.WLST repositoryCreator.py
  Initializing WebLogic Scripting Tool (WLST) ...
  Welcome to WebLogic Server Administration Scripting Shell
  Type help() for help on available commands
  Connecting to t3://XXXXXXX:ZZZZ with userid weblogic ...
  Successfully connected to Admin Server 'AdminServer' that belongs to domain 'bifoundation_domain'.
  Warning: An insecure protocol was used to connect to the
  server. To ensure on-the-wire security, the SSL port or
  Admin port should be used instead.
  Connecting to Domain ...
  Location changed to domain custom tree. This is a writable tree with No root.
  For more help, use help(domainCustom)
  Connecting to BIDomain MBean ...
  No stack trace available.
  Disconnected from weblogic server: AdminServer
  

• Restart the BI Server

Inference

• There is no limit set for Repository Size
• Online Access is bound to be slower for bigger sized repositories
• Smaller Sizes Repositories are easy to manage and develop
• The Operating System Limit is too big to worry about
• Enterprise Manager has work around to load bigger size repositories
• Manual Load is recommended for bigger repository
• It is a shame that Oracle does not offer any solution around multiple repositories within one BI Domain