datapump expdp without access to expdp tool nor filesystem

This article is about getting a database export dump without having access to the expdp tool nor the filesystem. In some cases you just don’t have access to the databaseserver and no one is available (like a dba), still you need sufficient accessrights. Note: this is merely an excersize, if you’re planning on doing this notify your dba anyway, this article is not about secretly getting a database dump :-)

For this to work, you don’t need access to the expdp tool or the filesystem but you need some rights on a directory object (preferable read/write) and have execute rights on the DBMS_DATAPUMP package.

Every step is mentioned here, maybe in your situation you can skip some steps.

1. Create a test user

CREATE USER TSTUSER
 IDENTIFIED BY "testing"
 DEFAULT TABLESPACE USERS
 TEMPORARY TABLESPACE TEMP
 ACCOUNT UNLOCK;

GRANT UNLIMITED TABLESPACE TO TSTUSER;
GRANT CREATE SESSION TO TSTUSER;
GRANT CREATE TABLE TO TSTUSER;
GRANT EXECUTE ON SYS.DBMS_DATAPUMP TO TSTUSER;
GRANT EXP_FULL_DATABASE TO TSTUSER;

2. Create a directory object where the output will get written to

CREATE OR REPLACE DIRECTORY DATADIR_FLATF AS '[D:\directorypath]';
GRANT READ, WRITE ON DIRECTORY DATADIR_FLATF TO TSTUSER;

3. Create a controltextfile. In this file the filenames should be present which represent your dump files. Save your controlfile ‘lob_data.txt’ to the DATADIR_FLATF location.

DECLARE
  out UTL_FILE.FILE_TYPE;
BEGIN
  out := UTL_FILE.FOPEN('DATADIR_FLATF', 'lob_data.txt' , 'W');
  UTL_FILE.PUT_LINE(out , '1,TWO_SCHEMAS.DMP,');
  UTL_FILE.PUT_LINE(out , '2,TWO_SCHEMAS.LOG,');
  UTL_FILE.FCLOSE(out);
END;
/

content of file is now (extra , at the end is not a typo):
1,TWO_SCHEMAS.DMP,
2,TWO_SCHEMAS.LOG,

4. Create a datapump export using DBMS_DATAPUMP

DECLARE
  handle NUMBER;
  status VARCHAR2(20);
BEGIN
  handle := DBMS_DATAPUMP.OPEN ('EXPORT', 'SCHEMA');
  DBMS_DATAPUMP.ADD_FILE (handle, 'TWO_SCHEMAS.DMP', 'DATADIR_FLATF');
  DBMS_DATAPUMP.ADD_FILE (handle, 'TWO_SCHEMAS.LOG', 'DATADIR_FLATF', null, DBMS_DATAPUMP.ku$_file_type_log_file);
  DBMS_DATAPUMP.METADATA_FILTER (handle, 'SCHEMA_EXPR', 'IN (''SCHEMA1'',''SCHEMA2'')');
  DBMS_DATAPUMP.START_JOB (handle);
  DBMS_DATAPUMP.WAIT_FOR_JOB (handle, status);
  DBMS_OUTPUT.PUT_LINE('Status : ' || status);
END;
/

5. Create an external table to gain access

CREATE TABLE lob_tab (
  number_content    NUMBER(10),
  blob_content      BLOB
)
ORGANIZATION EXTERNAL
(
  TYPE    ORACLE_LOADER
  DEFAULT DIRECTORY DATADIR_FLATF
  ACCESS  PARAMETERS
  (
    RECORDS DELIMITED BY NEWLINE
    BADFILE DATADIR_FLATF:'lob_tab_%a_%p.bad'
    LOGFILE DATADIR_FLATF:'lob_tab_%a_%p.log'
    FIELDS  TERMINATED BY ','
    MISSING FIELD VALUES ARE NULL
    (
      number_content    CHAR(10),
      blob_filename     CHAR(100)
    )
    COLUMN TRANSFORMS (blob_content FROM LOBFILE (blob_filename) FROM (DATADIR_FLATF) BLOB)
  )
  LOCATION ('lob_data.txt')
)
PARALLEL 2
REJECT LIMIT UNLIMITED
/

6. Test query

SELECT number_content, DBMS_LOB.getlength(blob_content) AS blob_length
FROM   lob_tab;

7. If you encounter the following

ORA-12801: error signaled in parallel query server P000
ORA-29913: error in executing ODCIEXTTABLEFETCH callout
ORA-29400: data cartridge error
error opening file D:\directorypath\TWO_SCHEMAS.DMP

place extra , at the end of each line maybe?

8. Download the files to your local OS using your favorite tool

datapump with flatfile

And there you have it. A complete dump of your schema choice locally available without having the expdp tool installed on your system.

9. Note, if you do have access to the filesystem, it looks similar to this

datapump with flatfile 2

10. Drop tstuser and objects

DROP USER TSTUSER CASCADE;

 

Generate XSD from base table with foreign key to primary key references

For our Oracle BizTalk interface we use an XSD file to describe the master detail table sets. Surely there are enough tools to generate the XSD file, so here is another one which I created to fit our needs. It’s easily adjustable.

It starts from the base table, searches for foreign key references which map to the primary key using the following query:

    select dco1.owner
         , dco1.table_name
         , dcc.column_name
    from   dba_constraints  dco1
         , dba_cons_columns dcc
    where  dco1.constraint_type = 'R'
    and    dco1.r_constraint_name in (select dco2.constraint_name
                                      from   dba_constraints dco2
                                      where  dco2.owner      = '[OWNER]'
                                      and    dco2.table_name = '[TABLE]'
                                      and    dco2.constraint_type = 'P')
    and    dcc.owner = dco1.owner
    and    dcc.table_name = dco1.table_name
    and    dcc.constraint_name = dco1.constraint_name;

This way, other foreign keys to unique attributes are not included in the XSD file generation, just the one’s who refer to the primary key are included.

When references are found it calls itself and so on (a maximum depth can be supplied).

For the BizTalk implementation only NUMBER or STRING element types are relevant, but it is easy to adjust the attribute generation using the CASE statement:

    select column_name as element_name
         , case when dtc.data_type = 'NUMBER' then 'type="xs:int"/>' else 'type="xs:string"/>' end as element_type
    from   dba_tab_cols dtc
    where  dtc.owner      = '[OWNER]'
    and    dtc.table_name = '[TABLE]'
    order  by dtc.column_id;

The package generates a CLOB which you can insert into a table or write to a file. In the example the CLOB is inserted into a table:

CREATE TABLE TMP_CLOB (message CLOB, datetime_col DATE);

The package:
Package PCK_GenXsd.pks specification.
Package PCK_GenXsd.pkb body.

Call it:

begin
  PCK_GenXsd.PR_GenXsd('NICENAME','OWNER','STARTINGTABLE');
end;
/

Clipboard swap in Java (Ctrl+Exchange or Ctrl+Swap)

After some inspiring Friday afternoon discussion, the following came to mind as nice to have.

Wouldn’t it be nice to have a CTRL+C and CTRL+V in one action, meaning your selected text is put on the clipboard and the previous clipboard is put on screen. In other words, a CTRL+Exchange or CTRL+Swap.

Probably there are dozens of tools like this and surely relatively easy to create in Visual Basic, in C# or in any other Windows (scripting) language, fact is I’m not a Windows developer (yet ;). But with no administrator rights, no possibility to install anything (USB not working either) and a tough e-mail scanner, our options are limited. But “Oracle SQL Developer” is installed (including a JRE).

Secondly, this should work globally in Windows across applications so the system tray comes in mind. Unfortunately, access is denied (later I will create the same for an at home solution in the system tray for sure).
And third, the Java application (while minimized) should be able to capture a CTRL+[key] globally and then exchange it and paste it back (e.g. CTRL+Q).
At last, it should not be too difficult and time consuming, since we are a week before sprint delivery.

Luckily, I came across this nice (although very old) post by Sergei Biletnikov:
http://biletnikov-dev.blogspot.nl/2009/09/global-hotkeys-for-java-applications_25.html
in which Sergei proposes a solution for the global capture of hotkeys using a .dll in Java

Link to the .dll files still works:
https://code.google.com/p/jkeyboard-hook/downloads/list

Loading a .dll in Java is pretty easy, but it should be in one .jar file since the executable should be in the form of “java -jar clipboardexchanger.jar”. For the later-to-build-system-tray-version other options are available. But for now some more Java combining forces are required, this time with Adam Heinrich and:
https://github.com/adamheinrich/native-utils
which allows us to load a .dll from a classpath while developing and from inside a .jar file when deployed.

What should it do by example: ctrl-exchange-2

  • “hello” is currently in the clipboard
  • “world” is selected somewhere in Notepad
  • User presses CTRL+Q
  • Current clipboard is loaded to remember (“hello”)
  • Perform CTRL+C (“hello” is replaced by “world” in clipboard)
  • Current clipboard is loaded to remember (“world”)
  • Put remembered (“hello”) back into the clipboard
  • Perform CTRL+V to paste, user sees “hello”
  • Put remembered “world” into clipboard
  • Other applications can use CTRL+V to paste “world” as well

When you download Sergei’s example and .dll files and add Adam’s native utils to load the .dll, you can add the following code in the
onGlobalHotkeysPressed() method:

public void onGlobalHotkeysPressed() {
  // get our current clipboard content
  String current = getClipboardContents(false); // "hello"

  // perform CTRL+C to fill clipboard with selected text
  r.keyPress(KeyEvent.VK_CONTROL);
  r.keyPress(KeyEvent.VK_C);
  r.delay(50);
  r.keyRelease(KeyEvent.VK_C);
  r.keyRelease(KeyEvent.VK_CONTROL);

  // read the system clipboard again to get the selected text, "world" is now known inside our application
  String selected = getClipboardContents(false);

  // put "hello" back into clipboard to be able to perform CTRL+V on it
  setClipboardContents(current, false);

  // perfor CTRL+V to paste "hello"
  r.keyPress(KeyEvent.VK_CONTROL);
  r.keyPress(KeyEvent.VK_V);
  r.delay(50);
  r.keyRelease(KeyEvent.VK_V);
  r.keyRelease(KeyEvent.VK_CONTROL);

  // put originally selected "world" into clipboard
  setClipboardContents(selected, true);
}

With “r” earlier declared:

java.awt.Robot r
try {
  r = new Robot();
} catch (AWTException e) {
  System.out.println("AWT Robot error");
}

As you can see, I use an old java.awt.Robot class to perform system tasks. Well, why not.

The “getClipboardContents” and “setClipboardContents” are widely known to get and set the system’s clipboard. I added a boolean to the method just to indicate if they should do something to the system tray. They are not really necessary. The methods (simplified):

/**
* Get the clipboard.
*
* @return any text found on the Clipboard; if none found, return an empty String
*/
private static String getClipboardContents(boolean showResult) {
  String result = "";
  Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
  Transferable contents = clipboard.getContents(null);
  boolean hasTransferableText = (contents != null) && contents.isDataFlavorSupported(DataFlavor.stringFlavor);
  if (hasTransferableText) {
    try {
      result = (String) contents.getTransferData(DataFlavor.stringFlavor);
    } catch (UnsupportedFlavorException | IOException ex) {
      result = "";
      System.out.println(ex);
      ex.printStackTrace();
    }
  }
  if (showResult) {
    System.out.println("clipboard contains: " + result);
  }
  return result;
}
/**
* Place a String on the clipboard.
*/
private static void setClipboardContents(String str, boolean showResult) {
  StringSelection stringSelection = new StringSelection(str);
  Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
  clipboard.setContents(stringSelection, stringSelection);
  if (showResult) {
    System.out.println("putting in clipboard: " + str);
  }
}

Finally, you’ll have to create a .jar file. If you’re using an IDE this should be rather easy. After creating the .jar, just add the .dll to the directory where the classes are. The NativeUtils.loadLibraryFromJar method will search in those.

With the following files:
GlobalKeyboardHook.java
GlobalKeyboardListener.java
NativeUtils.java
ClipboardExchanger.java (main method)

Or just download the .jar files (32bit and 64bit versions) and run the inside a command prompt box:
java -jar clipboardexchanger32.jar
and
java -jar clipboardexchanger64.jar